Current User: Guest Login
Please consider registering


Lost Your Password?

Search Forums:


 






Minimum search word length is 4 characters – Maximum search word length is 84 characters
Wildcard Usage:
*  matches any number of characters    %  matches exactly one character

awstats won't fill the gaps

Reply to Post Add a New Topic
UserPost

10:47 am
April 17, 2009


jc

Guest

I've got the awstats[MM][YYYY].domain.txt files going back to July 2008. A problem means I'm missing between 27th Dec 2008 and 6th April 2009.

Having read about my problem I removed this month's file believing awstats would see the last stats file was 122008 and would parse the log file (that I've generated in the format specified in the config file for that period) and write new stats for 27th Dec onwards.

My problem is that the in log file I'm giving it contains approx 22000 records, it jumps 21500 of them and just adds the stats for the last 24 hours or so.

The only reference to the LastLine it's processed I could find is in the stats files, so I changed that to match the last record:

BEGIN_GENERAL 8
LastLine 20090406142201 21431 2574944 27607447828
FirstTime 20081201001045
LastTime 20081226235701
LastUpdate 20090417163025 1 0 0 0 0

changed to

BEGIN_GENERAL 8
LastLine 20081226235701 21431 2574944 27607447828
FirstTime 20081201001045
LastTime 20081226235701
LastUpdate 20081226235701 1 0 0 0 0

but I assume this is not the only record of the last record it keeps as if I run the update again with these changes, it still jumps the ~21500 records and writes the few hundred records for April to a new stats file.

Does anyone know where awstats keeps the critical date/time that is stopping the stats updating for the missing period?

Any help gratefully received!

jc

6:50 am
April 18, 2009


Jean-Luc

Admin

posts 1125

Hi,

Move all files from the DirData  directory to a temprorary directory, then run all the updates starting with the log file of December 1st. When this is all done, restore the AWStats database files of November 2008 and older.

3:13 am
April 20, 2009


jc

Guest

Thanks for you reply.  I've tried moving all of the files out of the DirData, but even then it still only generates stats from the point it is magically remembering it last processed for.  Any ideas?

thanks, jc

4:12 am
April 20, 2009


Jean-Luc

Admin

posts 1125

I don't believe in computer magics. Wink

Please follow these steps:

  1. make sure that DirData is empty.
  2. from the command line, run the update for December 1st only.
  3. paste here what you get on the screen when running the update.

4:29 am
April 20, 2009


jc

Guest

Thank you very much for the reply!

I have been running the following

/usr/bin/perl /data/log_services/awstats/wwwroot/cgi-bin/awstats.pl -config=maillogs.int.rroom.net -update -LogFile=/logs/mail/mail.log

but I can't see an option for the awstats.pl script to specify to run it for Dec 1st only, can you help me out with that too? Embarassed

4:40 am
April 20, 2009


Jean-Luc

Admin

posts 1125

I was assuming that you had one log file per day.

If you have a log file for several days or just a large log file for an undefined period, check that it includes valid data in a valid format for the period between December 1 and December 27 and just run the update for that log file. Then paste the screen output here.

4:54 am
April 20, 2009


jc

Guest

Ah ok, I see.  I'd just put a file together with all the data for the missing period.  I've grepped out everything for December into a separate log file and run it against that file:

bash-3.00# /usr/bin/perl /data/log_services/
awstats/wwwroot/cgi-bin/awstats.pl -config=maillogs.int.rroom.net -update -LogFile=/logs/mail/mail.log
Create/Update database for config “/etc/awstats/awstats.maillogs.int.rroom.net.conf” by AWStats version 6.7 (build 1.892)
From data in log file “/logs/mail/mail.log”…
Phase 1 : First bypass old records, searching new record…
Searching new records from beginning of log file…
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)…
Jumped lines in file: 0
Parsed lines in file: 715
 Found 0 dropped records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 715 new qualified records.

However, now if I reload December in my browser I just get what I assume is a progress .gif (spinning arrows) and no stats.  the awstats122008 file also appears to contain no data for any actual emails.

Perhaps then it is my log format?  Here's what's in the config file, created the stats upto December and is now generating stats since early April:

LogFormat=”%time2 %host $host_r %logname %bytesd %email %email_r %url %method %code”

and here is an example line from my log file I'm trying to process

2008-12-31 14:05:55 62.128.140.97 domain.com NOAUTH 1914 website@domain.com somebody@yahoo.co.uk 1LI1hl-0007bE-T6 SMTP 250

Thanks, jc

5:28 am
April 29, 2009


jc

Guest

Thanks for the replies, you're right not to believe in computer magic.  The reason awstats didn't like my logs was a tab had crept onto the end of every line in the log file.  When I removed these the stats went in just fine.

thanks again, jc

Reply to Post

Reply to Topic:
awstats won't fill the gaps

Guest Name (Required):

Guest Email (Required):

NOTE: New Posts are subject to administrator approval before being displayed

Smileys
Confused Cool Cry Embarassed Frown Kiss Laugh Smile Surprised Wink Yell
Post New Reply

Guest URL (required)

Math Required!
What is the sum of:
10 + 4
   


About the InternetOfficer.com Forum

Forum Timezone: UTC 1

Most Users Ever Online: 302

Currently Online:
8 Guests

Currently Browsing this Topic:
1 Guest

Forum Stats:

Groups: 2
Forums: 9
Topics: 638
Posts: 2712

Membership:

There are 257 Members
There have been 305 Guests

There is 1 Admin
There is 1 Moderator

Top Posters:

cssfsu – 55
deepakgupta – 34
albert_newton – 30
cosminpana – 20
DTNMike – 19
ahtshun83 – 17

Recent New Members: raju, todd2taylor, sbdcunha, mansigill1987, ThomasDuh, ThomasKic

Administrators: Jean-Luc (1125 Posts)

Moderators: Jean-Luc (1125 Posts)