Current User: Guest Login
Please consider registering


Lost Your Password?

Search Forums:


 






Minimum search word length is 4 characters – Maximum search word length is 84 characters
Wildcard Usage:
*  matches any number of characters    %  matches exactly one character

Problem with logresolve.pl with two types logs

Reply to Post Add a New Topic
UserPost

12:25 am
August 8, 2010


geemist

Member

posts 7

Hello,

 

I have one file localhost.conf with multiple log files which i would like run my awstats but my log files are of a different format, how make with logresolve.pl ?

I have one log file which i want analyse :

/var/log/httpd/access_log

and i have a directory with several file PC1-01.gz, PC1-02.ge, PC2-01.gz … which i want analyse too.

In my localhost.conf, i have:

LogFile="/var/log/httpd/access_log"

#LogFile="gzip -cd /logs/PC1-* |"

LogFile="/usr/share/awstats/tools/logresolve.pl gzip -cd /logs/PC1-* /var/log/httpd/access_log > /tmp/temp.log |"

LogFormat=1

For the moment the logs are of the same type because i proceed step by step but if my logs are of one different format type, how do you make ?

When i run my command awstats.pl, i have a problem with logresolve.pl because it don't distinguish all my logs.

Help please

Thanks you very much

7:21 am
August 8, 2010


Jean-Luc

Admin

posts 1125

Hi,

I have no magic solution here. If you need to merge log files of totally different formats, you have to convert these files to a common format before you pass them to logresolvemerge.

Wherever possible, I would prefer to configure the servers so that they generate the logs in the same format anyway.

11:12 am
August 8, 2010


geemist

Member

posts 7

Ok thanks,

 

It's possible to handle several logs.gz with logresolve.pl from different directory ? How ?

For example, i have two directories:

 

ls /log/dir1:

pc1_2010_08_01_.gz

pc1_2010_08_02_.gz

pc1_2010_08_01_.gz

 

ls /log/dir2

pc2_2010_08_01_.gz

pc2_2010_08_02_.gz

pc2_2010_08_01_.gz

 

That's not running:

 LogFile="/usr/share/awstats/tools/logresolve.pl gzip -cd /logs/dir1/PC1-* /logs/diir2/PC1-* |"

 or

 LogFile="/usr/share/awstats/tools/logresolve.pl gzip -cd /logs/dir1/PC1-*  gzip -cd /logs/diir2/PC1-* |"

11:31 am
August 8, 2010


Jean-Luc

Admin

posts 1125

I would try this:

LogFile="/usr/share/awstats/tools/logresolvemerge.pl /logs/dir1/pc1_* /logs/dir2/pc2_* |"

.gz  files are directly supported by logresolvemerge.pl .

12:10 pm
August 8, 2010


geemist

Member

posts 7

That's good, it's running.

Thanks you Jean-Luc

8:19 pm
August 12, 2010


geemist

Member

posts 7

Is Awstats_buildstaticpages.pl able to accumulate statistics / reports of previous months although it only generates reports for a single month? If yes, should I only specify the same path for the "-dir"  option with awstats_buildstaticpages.pl ?

I have another problem: I want to generate AWStats reports on the 30 or 31 of the months with two different configuration files, as you noticed me Jean-Luc,
thank you.

Therefore I don't use logresolveme.pl but instead I have this in one of my conf files:

LogFile = "gzip -cd /var/log/dir1/MM/file*-YYYY-MM-DD.gz |"

My files are all in the same format, for example:
file01-YYYY-MM-DD.gz
file02-YYYY-MM-DD.gz
file03-YYYY-MM-DD.gz

I would like to either configure my logFile they take only the files of the current month or either configure my logFile in order to take all the files in a directory format "YYYY-MM" that I could create before.
I don't know if both solutions are possible?

What do you recommend me?

Thank you

9:47 pm
August 12, 2010


Jean-Luc

Admin

posts 1125

Okay, I will try to briefly explain a few tihings about AWStats:

  • when you run an AWStats update, some log files are read and the AWStats database files are updated
  • from the AWStats database files, you can dynamically generate dynamic report with awstats.pl
  • from the AWStats database files, you can also generate files containing the static HTML reports
  • there is one file per month in the AWStats database

So, if you want static files (HTML reports),  you have two major steps:

  1. data in log files => build the database files
  2. database files => build the HTML reports

Step 1 must be run chronologically (you cannot go back in the time). Your LogFile directive must specify the appropriate files.

Step 2 can be run at any time to generate the reports for any month.

Note that, in some cases, buildstaticpages can perform both steps with one command.

Please read the part about LogFormat in the awstats.model.conf file where they explain the %YYYY, %MM and %DD parameters. Also read the section Building and reading reports in AWStats documentation.

If you have two sets of log files and if you want two sets of reports, you need to use two different AWStats config files.

Do not to try to build a complex setup before you have successfully build less complicated configurations. Solving one problem at a time will probably be more effective.

11:58 pm
August 12, 2010


geemist

Member

posts 7

I dont' know if I understood everything but I cannot understand why you say "can not go back in the time". That means I could never generate reports for the month of July 2009 because they were not treated in July 2009 by awstats.pl that means they are not in the database files?

So what is the interest in options to specify LogFormat YYYY MM DD ? That is only if he event logs are already in the database? "

Is what I can about a file awstats.localhost.html reports make visible the entire year and on logs generated since a year but never treated so far by awstats and therefore not stored in the database awstats ?

Can I generate reports on one year with awstatsbuildstaticpages.pl on a single report awstats.localhost.html ?

I have looked awstats.pl

I quote: "If you want to build a report for a Particular month, add the options-month = MM-year = YYYY"

it becomes blurred with what you said: "(You Can not Go Back In The Time)" ????

 

It's not clear but it's becomes

Thanks

10:09 am
August 13, 2010


Jean-Luc

Admin

posts 1125

When I said "you cannot go back in the time", I meant that you cannot generate the database files for June after you have generated the database files for July or August.

Assume you just installed AWStats today. Then you have to generate the database files starting from the oldest log file you have. If it is March 2010, you have to run the AWStats updates (= generate the database files) for March, then April, then May, June, July,… You cannot run the AWStats update of July first and then June or May.

I hope this is more clear.Wink

Note that this only applies to the generation of the AWStats database files. You can generate the static HTML reports whenever you want.

10:36 am
August 13, 2010


geemist

Guest

Ok I think that more clear for me, but what program that generates the database files? It's awstats.pl?

How empty the database in order to generate the database of old log files in March, for example? Or I must desinstall awstats and then reinstall it ?

 

However, You don't answer to my principal question:

My files are all in the same format, for example:

 

How configure my LogFile:

LogFile = "gzip -cd /var/log/dir1/MM/file*-YYYY-MM-DD.gz *|"

11:28 am
August 13, 2010


Jean-Luc

Admin

posts 1125

Let us take it point by point:

1. You generate database files with AWStats update commands, like

./awstats.pl -config=my_domain – update

2. The database files are in the DirData  directory (defined in the AWStats config file). To empty this directory, delete its content or move it to another directory.

3.

LogFile = "gzip -cd /var/log/dir1/MM/file*-%YYYY-%MM-*.gz |"

will parse all files of this month.

LogFile = "gzip -cd /var/log/dir1/MM/file*-2010-05-*.gz |"

will parse all files of May 2010.

LogFile = "gzip -cd /var/log/dir1/MM/file*.gz |"

will parse all files.

10:40 pm
August 14, 2010


geemist

Member

posts 7

Thanks you Jean Luc !

Reply to Post

Reply to Topic:
Problem with logresolve.pl with two types logs

Guest Name (Required):

Guest Email (Required):

NOTE: New Posts are subject to administrator approval before being displayed

Smileys
Confused Cool Cry Embarassed Frown Kiss Laugh Smile Surprised Wink Yell
Post New Reply

Guest URL (required)

Math Required!
What is the sum of:
9 + 10
   


About the InternetOfficer.com Forum

Forum Timezone: UTC 1

Most Users Ever Online: 302

Currently Online:
21 Guests

Currently Browsing this Topic:
1 Guest

Forum Stats:

Groups: 2
Forums: 9
Topics: 639
Posts: 2710

Membership:

There are 257 Members
There have been 304 Guests

There is 1 Admin
There is 1 Moderator

Top Posters:

cssfsu – 55
deepakgupta – 34
albert_newton – 30
cosminpana – 20
DTNMike – 19
ahtshun83 – 17

Recent New Members: raju, todd2taylor, sbdcunha, mansigill1987, ThomasDuh, ThomasKic

Administrators: Jean-Luc (1125 Posts)

Moderators: Jean-Luc (1125 Posts)