Current User: Guest Login
Please consider registering


Lost Your Password?

Search Forums:


 






Minimum search word length is 4 characters – Maximum search word length is 84 characters
Wildcard Usage:
*  matches any number of characters    %  matches exactly one character

logresolvemerge keeping duplicates

Reply to Post Add a New Topic
UserPost

9:57 pm
March 18, 2010


cssfsu

Member

posts 55

Hi jean,

I want to use logresolvemerge for merging a few log files together. However I see that it does not get rid of duplicates. I know its only a tool for merging, but is there any way that I can have logresolvemerge get rid of duplicates ?

For eg log1.log has

9.49.199.111 – - [04/Feb/2010:18:33:41 -0500] "GET / HTTP/1.1" 302 2296 "18734/CF-WRK: 982399630 : 0" 3943591
9.49.199.111 – - [04/Feb/2010:18:33:42 -0500] "GET /login.gt HTTP/1.1" 200 14400 "18734/CF-WRK: 982399630 : 0" 700701
9.49.199.111 – - [04/Feb/2010:18:33:44 -0500] "GET /js/cic.js HTTP/1.1" 302 2296 "18734/CF-WRK: 982399630 : 1" 17234

log2.log has

9.49.199.111 – - [04/Feb/2010:18:33:41 -0500] "GET / HTTP/1.1" 302 2296 "18734/CF-WRK: 982399630 : 0" 3943591
–something new here—

The combined log shows the first entry from log2.log as well (which is a dup)

9.49.199.111 – - [04/Feb/2010:18:33:41 -0500] "GET / HTTP/1.1" 302 2296 "18734/CF-WRK: 982399630 : 0" 3943591
9.49.199.111 – - [04/Feb/2010:18:33:41 -0500] "GET / HTTP/1.1" 302 2296 "18734/CF-WRK: 982399630 : 0" 3943591 9.49.199.111 – - [04/Feb/2010:18:33:42 -0500] "GET /login.gt HTTP/1.1" 200 14400 "18734/CF-WRK: 982399630 : 0" 700701
9.49.199.111 – - [04/Feb/2010:18:33:44 -0500] "GET /js/cic.js HTTP/1.1" 302 2296 "18734/CF-WRK: 982399630 : 1" 17234
–something new here—

10:06 pm
March 18, 2010


cssfsu

Member

posts 55

One more question: If the combined log does have duplicates, how will Awstats process it as? Will it count the below as 2 hits or is it smart enough to know duplicates and count as 1 ?

9.49.199.111 – – [04/Feb/2010:18:33:41 -0500] "GET / HTTP/1.1″ 302 2296 "18734/CF-WRK: 982399630 : 0″ 3943591
9.49.199.111 – – [04/Feb/2010:18:33:41 -0500] "GET / HTTP/1.1″ 302 2296 "18734/CF-WRK: 982399630 : 0″ 394359

9:19 am
March 20, 2010


Jean-Luc

Admin

posts 1125

Hi,

Duplicates will count as 2 hits. logresolvemerge.pl  does not remove them either.

You will have to clean up the log files before you pass them to AWStats if you want to get rid of the duplicates. This will require to patch logresolvemerge.pl  or awstats.pl  or to parse the log file one more time, before it is processed by awstats.pl .

Reply to Post

Reply to Topic:
logresolvemerge keeping duplicates

Guest Name (Required):

Guest Email (Required):

NOTE: New Posts are subject to administrator approval before being displayed

Smileys
Confused Cool Cry Embarassed Frown Kiss Laugh Smile Surprised Wink Yell
Post New Reply

Guest URL (required)

Math Required!
What is the sum of:
11 + 6
   


About the InternetOfficer.com Forum

Forum Timezone: UTC 1

Most Users Ever Online: 302

Currently Online:
7 Guests

Currently Browsing this Topic:
1 Guest

Forum Stats:

Groups: 2
Forums: 9
Topics: 639
Posts: 2710

Membership:

There are 257 Members
There have been 304 Guests

There is 1 Admin
There is 1 Moderator

Top Posters:

cssfsu – 55
deepakgupta – 34
albert_newton – 30
cosminpana – 20
DTNMike – 19
ahtshun83 – 17

Recent New Members: raju, todd2taylor, sbdcunha, mansigill1987, ThomasDuh, ThomasKic

Administrators: Jean-Luc (1125 Posts)

Moderators: Jean-Luc (1125 Posts)