Current User: Guest Login Register
Please consider registering


Register? | Lost Your Password?

Search Forums:


 






Minimum search word length is 3 characters – Maximum search word length is 84 characters
Wildcard Usage:
*  matches any number of characters    %  matches exactly one character

Parsing IIS W3C Extended …

Reply to Post Add a New Topic
UserPost

3:17 am
January 22, 2008


Kris

Guest

Hello,

I installed AWStat today, I checked the procedure and no problem. But in this procedure, we have to change the log format, the "old" log are not more usable to generate stat. Then I switched back to the original parameter of IIS log :

W3C Extended + properties advanced (Date, Time, c-ip, cs-username, s-sitename, s-ip, s-port, cs-method, cs-uri-sterm, cs-uri-query, sc-status, sc-substatus, sc-win32-status, cs-user-agent)

Sample, extract from my real log file (I just replaced the ip by XXX and YYY) :

2008-01-22 08:48:16 W3SVC1139885586 xxx.XXX.XXX.XXX GET /default.aspx – 80 – 85.YYYY.YYYY.YYY Mozilla/5.0+(Windows;+U;+Windows+NT+6.0;+en-US;+rv:1.8.1.11)+Gecko/20071127+Firefox/2.0.0.11 200 0 0 

I tried to use your LogFormat found here : http://www.internetofficer.com/awstats/log-format/ 

But they don't work in my case.

Do you have others suggestions ?

Thanks for you help,

 

 

3:37 am
January 22, 2008


Jean-Luc

Admin

posts 1063

Please copy here the line from your log file starting with :

#Fields:

Jean-Luc

4:24 am
January 22, 2008


Kris

Guest

#Fields: date time s-sitename s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status

 

Regards, 

4:42 am
January 22, 2008


Jean-Luc

Admin

posts 1063

Your format is not compatible with AWStats. AWStats requires a %bytesd  field (called sc-bytes  by IIS). This field is the number of data bytes sent by the server to the client in each hit.

You can try to use the following LogFormat on your existing log file:

LogFormat="%time2 %other %other %method %url %query %other %logname %host %ua %code %other %bytesd"

You will get AWStats reports based on your log files, but all figures about bandwidth and number of bytes will be wrong.

Jean-Luc

7:52 am
January 22, 2008


Kris

Guest

Thanks Jean-Luc that's work fine :)

A small think when I execute this :

awstats.pl -config=myconfig_file -update 

and after this :

awstats.pl -config=myconfig_file -output -staticlinks > awstats.mysite.html  

In the HTML file I see only information from today and not from the begin of the month (web site started in january). Is it possible to launch a "full scan" for the directory log of this web site ?

 Thanks, 

8:29 am
January 22, 2008


Jean-Luc

Admin

posts 1063

Have a look at logresolvemerge.pl. It can be used to merge all your log files. Don't forget to remove the existing AWStats database file for January (with today's data) before you start it.

Jean-Luc

1:18 am
February 14, 2008


Bindu Ramunoju

Guest

how can i use awstats for following log format??

#Fields: time c-ip cs-method cs-uri-stem sc-status

 i have logs in this format from 2007 and i'm configuring awstats now.

so how can i analyze old logs?????

 thanks in advance for any idea

 

Regards,

Bindu Ramunoju

2:55 am
February 14, 2008


Jean-Luc

Admin

posts 1063

Same answer as above:

Your format is not compatible with AWStats. AWStats requires a %bytesd  field (called sc-bytes  by IIS). This field is the number of data bytes sent by the server to the client in each hit.

Does your "time" field include the date too ?

Anyway, you need to modify the file to make it compatible before you pass it to AWStats. I believe two changes might be required:

- time  field must become "date time";

- a "simulated" sc-bytes  field must be added.

Jean-Luc

3:13 am
February 14, 2008


Bindu Ramunoju

Guest

Thankq

Mr.Jean-Luc

ok tell me one thing more,

Now i can change my log to include %bytesd

but wht about the existing logs????

how can i change existing logs

if existing logs are not in format required by the awstats ,is it that awstats doesn't analyze them….

so it will only work for future logs not the existing Is it that??

 

Thanks in advance

Bindu Ramunoju

4:32 am
February 14, 2008


Jean-Luc

Admin

posts 1063

AWStats cannot work on your existing logs, but you can edit your existing logs to make them compatible. For example, to simulate an sc-bytes  field, you should add " 100" at the end of each line. To do that, use a utility like sed. More info about sed for Windows is available from sed . . . the stream editor.

12:01 am
February 15, 2008


Bindu Ramunoju

Guest

Thanks a lot for your support

1:29 am
June 24, 2008


Patrick

Guest

reading my IIS logs i have this:

#Date: 2008-05-01 04:00:00
#Fields: date time s-sitename s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken

 LogFormat = "date time s-sitename s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken"

giving me new records but have a lots of Unknown Errors. any idea what LogFormat should i put? Thanks in Advance

4:15 am
June 24, 2008


Jean-Luc

Admin

posts 1063

Try this:

LogFormat = "%time2 %other %other %method %url %query %other %logname %host %other %ua %other %referer %other %code %other %other %bytesd %other %other"

4:30 am
June 24, 2008


Patrick

Guest

thanks a lot.. cheers for you Jean-Luc!

6:48 pm
August 28, 2008


saurabh

Guest

HI Folks

Please help me here , im in big time trouble with IIS Extended W3c Format .

Here is what the Log File Format Says :

#Fields: date time c-ip cs-username s-ip s-port cs-method cs-uri-stem cs-uri-query sc-status cs(User-Agent)

and this is some of the actual Log File Entries

2008-07-28 00:11:34 3.44.126.104 501355027 3.44.150.136 80 GET /Programscheduling.aspx SMTRANSACTIONID=88962c03-0838-488d0eb5-0e74-01a804ea 200 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0;+.NET+CLR+1.1.4322)
2008-07-28 00:11:34 3.44.126.104 501355027 3.44.150.136 80 GET /Progjmp.aspx UserName=NBCTest2&ntgp=1&Module=Title%20Management&scrollenable=no&SMTRANSACTIONID=88962c03-0838-488d0eb6-0e74-01d65414 200 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0;+.NET+CLR+1.1.4322)
2008-07-28 00:11:34 3.44.126.104 501355027 3.44.150.136 80 GET /SearchTitles.aspx ntgp=1&SMTRANSACTIONID=88962c03-0838-488d0eb6-0bf0-02444eb7 200 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0;+.NET+CLR+1.1.4322)

I tried using this Log Format in Config File

LogFormat = “%time2 %host %logname %other %other %method %url %query %code %bytesd %ua”

It Didn't give me a error while using awstats.pl but there is no data that i see in the report and on the COnsole i got this message

Searching new records from beginning of log file…
Jumped lines in file: 0
Parsed lines in file: 63992
 Found 1 dropped records,
 Found 63991 corrupted records,
 Found 0 old records,
 Found 0 new qualified records.

Please Help me , please

2:01 am
August 29, 2008


Jean-Luc

Admin

posts 1063

Hi,

The problem is that your log file does not include the %bytesd  field (sc-bytes in IIS terminology). This field is mandatory for AWStats. The best thing to do is to add this field to your IIS log format.

For your existing log files, you can try this:

LogFormat = "%time2 %host %bytesd %other %other %method %url %query %code %ua"As this uses a dummy %bytesd , AWStats data related to bandwidth and number of bytes will be wrong, but all other information should be valid.

2:47 pm
September 16, 2008


Saurabh

Guest

Okay here is one more problem

These are some of the Log Files Entries which i have . i dont know which parameter to set for them :

2008-07-16 03:48:40 W3SVC334063184
USECLPLBW074WSA 64.210.198.30 GET /
default.asp – 80 – 75.47.246.136 HT
TP/1.1 Mozilla/4.0+(compatible;+MSI
E+6.0;+Windows+NT+5.1;+SV1;+.NET+CL
R+1.1.4322) setrentals.nbcuni.com 2
00 0 0 17003 312 218
2008-07-16 03:48:40 W3SVC334063184
USECLPLBW074WSA 64.210.198.30 GET /
styles.css – 80 – 75.47.246.136 HTT
P/1.1 Mozilla/4.0+(compatible;+MSIE
+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR
+1.1.4322) setrentals.nbcuni.com 20
0 0 0 4070 332 0
2008-07-16 03:48:40 W3SVC334063184
USECLPLBW074WSA 64.210.198.30 GET /
popup_calendar.js – 80 – 75.47.246.
136 HTTP/1.1 Mozilla/4.0+(compatibl
e;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.
NET+CLR+1.1.4322) setrentals.nbcuni
.com 200 0 0 5469 339 109

and the Log Format i used to process is

LogFormat = “%time2 %host %bytesd %other %other %method %url %query %code %ua”

Please help me out .. and if you could tell me how shall i create a log format for future just by seeing the logs that will be a big help

Waiting for the reply

Thnx..saurabh

3:06 pm
September 16, 2008


Jean-Luc

Admin

posts 1063

It is much easier to find the LogFormat with the content of the line starting with #Fields:

Our How To's : Log Format explains how to convert IIS log format codes into AWStats log format.

Please paste here the line starting with#Fields:  and I willl answer with the AWStats LogFormat definition.

3:40 pm
September 16, 2008


Saurabh

Guest

Ok this is the #field parameter

#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken

3:54 pm
September 16, 2008


Jean-Luc

Admin

posts 1063

Try this:

LogFormat = "%time2 %other %other %other %method %url %query %other %logname %host %other %ua %other %code %other %other %bytesd %other %other"

Reply to Post

Reply to Topic:
Parsing IIS W3C Extended …

Guest Name (Required):

Guest Email (Required):

NOTE: New Posts are subject to administrator approval before being displayed

Smileys
Confused Cool Cry Embarassed Frown Kiss Laugh Smile Surprised Wink Yell
Post New Reply

Guest URL (required)

Math Required!
What is the sum of:
4 + 1
   


About the InternetOfficer.com Forum

Forum Timezone: UTC 1

Most Users Ever Online: 201

Currently Online:
10 Guests

Currently Browsing this Topic:
1 Guest

Forum Stats:

Groups: 2
Forums: 9
Topics: 513
Posts: 2436

Membership:

There are 4127 Members
There have been 254 Guests

There is 1 Admin
There is 1 Moderator

Top Posters:

cssfsu – 55
albert_newton – 30
deepakgupta – 28
cosminpana – 20
DTNMike – 19
ahtshun83 – 17

Recent New Members: eniko, leptserkhan, JohnnyB, dweb3d, buck, newtoawstat

Administrators: Jean-Luc (1063 Posts)

Moderators: Jean-Luc (1063 Posts)