| User | Post |
|
5:45 am November 5, 2007
| Jim
Guest
| | | |
|
| |
|
|
Hi
We've recently started using Awstats to analyse logfiles on our clustered servers. All is working fine except when the merged log file from the 2 servers is created using the logresolve tool I have to paste the following at the top of the merged log file for Awstats to proces the log:
#Software: Microsoft Internet Information Services 6.0
#Version: 1.0
#Fields: date time cs-method cs-uri-stem cs-uri-query cs-username c-ip cs-version cs(User-Agent) cs(Referer) sc-status sc-bytes
This means that I cannot set up a scheduled task to run the job automatically, which is a shame!
Has anyone any ideas which might help?
Cheers in advance
Jim
|
|
|
8:01 am November 5, 2007
| Jean-Luc
Admin
| | | |
|
| posts 220 |
|
|
Hi Jim,
You probably use:
LogFormat=2
I would rather use this LogFormat :
LogFormat = "%time2 %method %url %query
%logname %host %other %ua %referer %code
%bytesd"
With the short form of the LogFormat, AWStats has to search for a format specification within the input file; it could be that logresolve removes this format specification.
Jean-Luc
|
|
|
9:27 am November 5, 2007
| Jim
Guest
| | | |
|
| |
|
|
Jean-Luc
Thank you very much for your quick reply!
This has done the trick!
I no longer need to paste that information into the combiuned log which means I can now automate the process and save myself lots of time.
Thank you very much for your help, I'm very grateful.
Cheers
Jim
|
|
|
3:43 am November 6, 2007
| Jim
Guest
| | | |
|
| |
|
|
Ooooopppsss!
Spoke too soon!
I've changed logfile format from LogFormat=2 to:
LogFormat = "%time2 %c-ip %cs-username %cs-method %cs-uri-stem %cs-uri-query %sc-status %sc-bytes %cs-version %cs(User-Agent) %cs(Referer)
Which is the format the logfiles are saved in. When I run Awstats now, it parses without error but doesn't produce any results when I creat a report. So I no longer need to paste that text into the top of the combined logfile like I did, which is great, but now I get no figures…confused?….oh yes!
Jim
|
|
|
7:11 am November 6, 2007
| Jean-Luc
Admin
| | | |
|
| posts 220 |
|
|
You write that you use:
LogFormat = "%time2 %c-ip %cs-username %cs-method %cs-uri-stem %cs-uri-query %sc-status %sc-bytes %cs-version %cs(User-Agent) %cs(Referer)"
This LogFormat does not match the fields specification in your first post:
#Fields: date time cs-method cs-uri-stem cs-uri-query cs-username c-ip cs-version cs(User-Agent) cs(Referer) sc-status sc-bytes
For example, according to your first post, the field after the date and time is cs-method which AWStats calls %method.
Did you try the LogFormat I suggested ?
At the end of the database update, AWStats prints a report like this one:
Create/Update database for config "/path_to_awstats/awstats.my_domainconf" by AWStats version 6.7 (build 1.892)
From data in log file "/path_to_log_file/log_file_name"…
Phase 1 : First bypass old records, searching new record…
Searching new records from beginning of log file…
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)…
Jumped lines in file: 0
Parsed lines in file: 3693
Found 520 dropped records,
Found 12 corrupted records,
Found 0 old records,
Found 3161 new qualified records.
This information helps understand what happened when there is a problem.
Jean-Luc
|
|
|
8:47 am November 9, 2007
| Jim
Guest
| | | |
|
| |
|
|
Jean-Luc
Thank you! I used the logformat from the boxes ticked on the IIS settings rather than the format I originally posted. I've now used your suggestion and the files are parsed without a problem.
Cheers for your help!
One last question!
Now I want to automate the process using a scheduled task. I can set up a btach file with my Awstat commands and get the scheduled task to run but it seems logresolve doesn't like the format D:/LogFiles/W3SVC1725568973/ex%YY-0%MM-0%DD-24 to merge yesterday's file. It works if I specify the exacy logfile name e.g. ex071108.log.
Any ideas?
Jim
|
|
|
9:01 am November 9, 2007
| Jean-Luc
Admin
| | | |
|
| posts 220 |
|
|
In batch files, the percent sign is a special character.
Try to replace each percent sign by two percent signs, like this:
D:/LogFiles/W3SVC1725568973/ex%%YY-0%%MM-0%%DD-24
Jean-Luc
|
|
|
9:40 am November 9, 2007
| Jim
Guest
| | | |
|
| |
|
|
Hi Jean-Luc
Thank you again for your quick reply
this is an improvement but now the command is interpreted literally and I get the following error written in the combined log:
Error: Couldn't open log file "D:/LogFiles/W3SVC1725568973/ex%YY-0%MM-0%DD-24" : No such file or directory.
I'll keep messing!
Cheers dude
Jim
|
|
|
11:36 am November 9, 2007
| Jean-Luc
Admin
| | | |
|
| posts 220 |
|
|
Two questions:
1. What error message did you get first with D:/LogFiles/W3SVC1725568973/ex%YY-0%MM-0%DD-24" ?
2. What is the complete line calling logresolvemerge.pl in your batch file ?
Jean-Luc
|
|
|
5:57 am November 12, 2007
| Jim
Guest
| | | |
|
| |
|
|
Hi Jean-Luc
The first error message was:
Error: Couldn't open log file "D:/LogFiles/W3SVC1725568973/exMM-0/LogFiles/W3SVC1725568973/exMM-0DD-24.log" : No such file or directory.
This was when using the following CMD line in my batch file:
D:/website/applications/awstats-6.6/tools/logresolvemerge.pl D:/LogFiles/W3SVC1725568973/ex%YY-0%MM-0%DD-24.log Z:/LogFiles/W3SVC1725568973/ex%YY-0%MM-0%DD-24.log > /LogFiles/website-combined-logs/combinedlogs.log
Cheers
Jim
|
|
|
11:05 am November 12, 2007
| Jean-Luc
Admin
| | | |
|
| posts 220 |
|
|
Did you replace all six original percent signs (% => %%) in the command line ? The .bat file should contain:
D:/website/applications/awstats-6.6/tools/logresolvemerge.pl D:/LogFiles/W3SVC1725568973/ex%%YY-0%%MM-0%%DD-24.log Z:/LogFiles/W3SVC1725568973/ex%%YY-0%%MM-0%%DD-24.log > /LogFiles/website-combined-logs/combinedlogs.log
At execution time, this should give:
D:/website/applications/awstats-6.6/tools/logresolvemerge.pl D:/LogFiles/W3SVC1725568973/ex%YY-0%MM-0%DD-24.log Z:/LogFiles/W3SVC1725568973/ex%YY-0%MM-0%DD-24.log > /LogFiles/website-combined-logs/combinedlogs.log
And logresolvemerge.pl should then replace %YY-0, %MM-0,… by the appropriate values.
Can you try again and let me know how the command looks like at execution time (ie. when you run the batch file). Thank you.
Jean-Luc
|
|
|
3:33 am November 13, 2007
| Jim
Guest
| | | |
|
| |
|
|
Hi Jean-Luc
This is the line I used in the batch file:
D:/website/applications/awstats-6.6/tools/logresolvemerge.pl D:/LogFiles/W3SVC1725568973/ex%%YY-0%%MM-0%%DD-24.log Z:/LogFiles/W3SVC1725568973/ex%%YY-0%%MM-0%%DD-24.log > /LogFiles/website-combined-logs/combinedlogs.log
When this is run in the batch file the combined log has the following error:
Error: Couldn't open log file "D:/LogFiles/W3SVC1725568973/ex%YY-0%MM-0%DD-24.log" : No such file or directory.
The two %% symbols format the line correctly but logresolve inerprets ex%YY-0%MM-0%DD-24.log literally rather than replacing the values with figures.
The execution lasts for less than a second as the logfile can't be found so nothing is parsed.
Jim
|
|
|
4:08 am November 13, 2007
| Jean-Luc
Admin
| | | |
|
| posts 220 |
|
|
Jim,
It's getting clear ! The point is that logresolvemerge.pl is not processing file names like awstats.pl does.
You cannot use these %YY-xx%MM-xx%DD-xx with logresolvemerge.pl. I am afraid you will have to use tricks around the DATE function in the batch file to automate things as you want.
Also note that %YY-0%MM-0%DD-24 is not going to do what you expect. On January 1, 2008, %YY-0%MM-0%DD-24 will return 080131. You should use %YY-24%MM-24%DD-24 that would return 071231.
Jean-Luc
|
|
|
6:38 am November 13, 2007
| Jim
Guest
| | | |
|
| |
|
|
Cheers jean-Luc
Thanks very much for your help, you've been brilliant.
I'll have a good mess around with my batch file.
Take it easy dude
Jim
|
|