Current User: Guest Login
Please consider registering


Lost Your Password?

Search Forums:


 






Minimum search word length is 4 characters – Maximum search word length is 84 characters
Wildcard Usage:
*  matches any number of characters    %  matches exactly one character

AWStats Not returning all stats – running AWStats locally on downloaded logs

Reply to Post Add a New Topic
UserPost

10:58 am
February 19, 2010


cogwirrel

Member

posts 4

I recently set up awstats 6.2 according to this guide (http://www.lunarforums.com/web…..084.0.html) so that I could analyse logs that I have already downloaded from my server. I have two webservers which each create separate logs for the same site. Before installing AWStats I used WeblogExpert which could analyse logs from both webservers, and give me detailed statistics on everything from visits to the browsers used.

I heard that AWStats gives better, more detailed statistics and so thought I'd try it.

In order for AWStats to look at my logs I used logresolvemerge.pl to merge daily logs from both webservers into one single huge 7GB log for the month:

@echo off

perl "C:awstats-6.2toolslogresolvemerge.pl" "C:awstats-6.2logs1*" > "C:awstats-6.2logsaccess.2010_01ws1.log"

perl "C:awstats-6.2toolslogresolvemerge.pl" "C:awstats-6.2logs2*" > "C:awstats-6.2logsaccess.2010_01ws2.log"

perl "C:awstats-6.2toolslogresolvemerge.pl" "C:awstats-6.2logsaccess.*" > "C:awstats-6.2logsaccess.2010_01.log"

echo Done

pause

As you can probably see, the batch file above merged all daily logs from webserver 1 into a monthly log, then did the same for webserver 2, then merged the two monthly logs to create a single log for both webservers for the month.

After this I ran a batch file which I thought would create all of the statistics I wanted:

@echo off
echo Generate Statistics?
pause
cd C:awstats-6.2wwwrootcgi-bin

perl awstats.pl -config=www.mywebsite.c
om -update

echo Done
echo Output Statistics?
pause

perl awstats.pl -config=www.mywebsite.c
om -output -staticlinks > awstats.www.m
ywebsite.com.html

perl awstats.pl -config=www.mywebsite.c
om -output=alldomains -staticlinks > aw
stats.www.mywebsite.com.alldomains.html

perl awstats.pl -config=www.mywebsite.c
om -output=errors404 -staticlinks > aws
tats.www.mywebsite.com.errors404.html

echo Done
echo View Statistics?
pause
call awstats.www.mywebsite.com.html

However this gave empty pages except for awstats.www.mywebsite.com.html, which only gave details on unique visitors, number of visits, pages, hits, bandwidth.

Every other field on this page contained no statistics, and pages such as awstats.www.mywebsite.com.browserdetail.html

Does anyone know how I can make AWStats return all of these statistics?

Does AWStats even work properly when I run the stats locally rather than on the servers?

Can AWStats handle logs from two webservers and am I dealing with the daily logs correctly by using logresolvemerge.pl to create this single huge log file for both webservers?

Thanks,

Jack

11:34 am
February 19, 2010


Jean-Luc

Admin

posts 1125

Hi,

All what you did seems correct to me, even if you did not get the expected result. There are better ways to do it. Here are my comments:

1. why use AWStats 6.2 when the latest release is AWStats 6.95 ?

2. there is no need to merge all daily logs files into a monthly log. AWStats can perfectly process the Jan 1 log file, then the Jan 2 log file, and so on.

3. if there are two log files for one site, you need to merge the logs, but you can just merge the daily log files. If your system supports pipes, you can use a pipe to pass the merged log file to AWStats so that there is no need for the space of the merged files on your disks. If you don't use pipes, you can delete the merged file as soon as AWStats has finished to process that day.

4. AWStats database files are created in the DirData  directory specified in your AWStats config file. Check that the AWStats database files are there. They are called awstatsMMYYYY.SITE-NAME.txt  where MM  is the month number, YYYY  is the year and SITE-NAME  is the name of the configuration (usually the server name).

5. after you created the AWStats database files, your batch file generates static output files containing the AWStats HTML reports for the current month. That is the reason why you do not see anything in the reports, if you collected the data about January and you generated the reports for February. To generate the HTML reports for January, you need to specify these options in all commands that generate output files (the ones that includes the -output  option): -month=01 -year=2010 .

Please let me know if this helps.

12:00 pm
February 19, 2010


cogwirrel

Member

posts 4

Hi,

I followed your instructions in point 5 and all of the information appeared! Thanks so much for your help! I can't believe it was something that simple!

I was only using 6.2 as that was what was used in the guide. I think now I will try to install 6.95 as I understand a bit more about it now and should be able to set it up locally.

I like to create a report monthly - if I did not merge the daily logs into a monthly log, can I update the statistics for all logs at the same time? Say, by specifying C:\awstats-6.2\logs\*.log in the config file? Or would I need to update the first log, change the config file, update the second, change the file again etc.?

The logs for webserver 1 and 2 have exactly the same name, is there a way to merge them together easily so that I have one set of daily logs rather than two? As with logresolvemerge.pl it seems like you can only select logs to merge in one directory, and the logs from webserver 1 and 2 must be kept separate due to their having the same name. Can you think of a batch file that will do this?

Thanks again for all your help,

Jack

1:08 pm
February 19, 2010


Jean-Luc

Admin

posts 1125

There are two steps in the all process:

  1. update of the AWStats database
  2. creation of the HTML reports

I would recommend that you update the AWStats database on a daily basis. You can run the update, say every day at 01:00 for the day before. This update demands a merge of the daily log files of the day before. You need logresolvemerge  for that. You can possibly do that within the LogFile  directive in the AWStats config file.

LogFile="perl C:pathlogresolvemerge D:path1file_%YY-24%MM-24%DD-24.log D:path2file_%YY-24%MM-24%DD-24.log |"

In this example, AWStats will merge two files before it runs the update of its database. %YY-24  and the following fields are date fields that are automatically calculated by AWStats as explained in the example AWStats config file included in the AWStats package. Of course, this is just an example that you have to modify according to your configuration.

You can run the database update daily and generate the HTML reports on a monthly basis if you wish.

1:23 pm
February 19, 2010


cogwirrel

Member

posts 4

Hi,

Thanks for your suggestion, however unfortunately I am not able to run the update each day – due to time constraints.

Am I able to use say:

LogFile="perl C:\awstats-6.95\tools\logresolvemerge.pl C:\awstats-6.95\logs\1\*.log C:\awstats-6.95\logs\2\*.log |"

To run an update for the entire month?

Thanks,

Jack

1:43 pm
February 19, 2010


Jean-Luc

Admin

posts 1125

This should work fine. It is important that the input files are only the log files of interest for the given period. This will avoid lost of time in reading again data which have already been processed in the past months.

1:55 pm
February 19, 2010


cogwirrel

Member

posts 4

Excellent. Thank you so much for all of this help – I couldn't have set it up without you! I'll give it a try and let you know if I have any problems.

Take it easy,

Jack

Reply to Post

Reply to Topic:
AWStats Not returning all stats – running AWStats locally on downloaded logs

Guest Name (Required):

Guest Email (Required):

NOTE: New Posts are subject to administrator approval before being displayed

Smileys
Confused Cool Cry Embarassed Frown Kiss Laugh Smile Surprised Wink Yell
Post New Reply

Guest URL (required)

Math Required!
What is the sum of:
5 + 1
   


About the InternetOfficer.com Forum

Forum Timezone: UTC 1

Most Users Ever Online: 302

Currently Online:
17 Guests

Currently Browsing this Topic:
1 Guest

Forum Stats:

Groups: 2
Forums: 9
Topics: 639
Posts: 2710

Membership:

There are 257 Members
There have been 304 Guests

There is 1 Admin
There is 1 Moderator

Top Posters:

cssfsu – 55
deepakgupta – 34
albert_newton – 30
cosminpana – 20
DTNMike – 19
ahtshun83 – 17

Recent New Members: raju, todd2taylor, sbdcunha, mansigill1987, ThomasDuh, ThomasKic

Administrators: Jean-Luc (1125 Posts)

Moderators: Jean-Luc (1125 Posts)