Current User: Guest Login
Please consider registering


Lost Your Password?

Search Forums:


 






Minimum search word length is 4 characters – Maximum search word length is 84 characters
Wildcard Usage:
*  matches any number of characters    %  matches exactly one character

Problem with Extra Saction

Reply to Post Add a New Topic
UserPost

1:06 pm
February 4, 2010


Deepak Gupta

Guest

Hi

I am using awstats 6.9 version. I am facing problems with Extra saction. In google searches I can see lots of entries as below

c2def087e4ae1e0c 1192 1192 34.32 MB 03 Feb 2010 – 23:57

When I click on this It takes me to google search and show the results which are not linked to my website any way. I am not able to find why these entries are appearing in search results and these kind of entries are too many.

Please help me to resolve the issue.

Deepak Gupta

1:54 pm
February 4, 2010


Jean-Luc

Admin

posts 1125

Hi,

Is http://www.google.com/search?q…..87e4ae1e0c the exact referrer as in the log file or did you build it within the extra section ? I ask, because search results can be different in google.com, in google.co.in, in google.fr and so on. They can also be different depending on other parameters in the URL's.

Note that a referrer can be forged by a browser or by a robot. In fact, you never know iif a referrer is real or dummy. Admittedly, most of the time, it is real.

Anyway I have a theory about it. Not sure though. This could be generated by a robot. The idea behind it could be that nobody blocks a visitor sent by Google. It could be used by people trying to automatically record many pages of your site: could be a competitor or just someone who wants to re-use your information in another application.

Look at the user agents and at the IP addresses that generate these hits and you will possibly better understand what happens. You can do that with addtional extra sections or by directly checking the log files.

1:17 pm
February 9, 2010


Deepak Gupta

Guest

Hi Jean-Luc

Thanks for your valuable feedback.

I did this in extra saction but using log files only. This kind of entries are not coming in referers. This entry is coming in google searches. I do agree with you that search result will be different for diff google servers but when I click on this value it takes me to same google sever.

My main concern is what are these entries. These are not searched keywords then why it is coming in google searches and that too its coming at no1 position. This is how it apprears as search keyword for google

Search  –   Full list Pages Hits Bandwidth Last visit
58ffa43166db4145 670 670 19.07 MB 09 Feb 2010 – 06:41

We are not getting any IP with this result how I can get IP of this search result.

1:31 pm
February 9, 2010


Jean-Luc

Admin

posts 1125

These entries do not come from the referrer section of the AWStats report, but they come from the referrer fields in the log file.

I suggest that you try to find these entries in the log file and that you paste a few of them here.

6:07 am
February 10, 2010


Deepak Gupta

Guest

Hi Jean

Thanks for your comments. Can u please explain how I can get IP address in extra saction as presently it is not shoqing any IP address of the referer

You can see we have only 5 fields in this section.

Search  –   Full list Pages Hits Bandwidth Last visit
58ffa43166db4145 670 670 19.07 MB 09 Feb 2010 – 06:41

7:09 am
February 10, 2010


Deepak Gupta

Guest

Hi Jean

As suggested I am pasting few logs with the keywords for your reference

****************************************************

124.123.242.221 – – [09/Feb/2010:23:11:
47 -0500] "GET /lat_long/andhrapradesh/
andhrapradesh.htm HTTP/1.1" 200 24133 "
http://www.google.co.in/search?hl=en&q=
hyderabad+latitude+longitude&meta=&aq=1
&oq=hyderabad+la&fp=12b81a5d82c30213" "
Mozilla/5.0 (Windows; U; Windows NT 6.0
; en-GB; rv:1.9.1.7) Gecko/20091221 Fir
efox/3.5.7 (.NET CLR 3.5.30729) FBSMTWB
"

119.82.94.132 – – [09/Feb/2010:22:35:46
 -0500] "GET /delhi/gagan-vihar.html HT
TP/1.1" 200 24267 "http://www.google.co
.in/search?hl=en&source=hp&q=gagan+viha
r+delhi&meta=&aq=0&oq=gagan+vihar&fp=44
adc7acd595e18a" "Mozilla/5.0 (Windows;
U; Windows NT 5.1; en-US; rv:1.9.1.7) G
ecko/20091221 Firefox/3.5.7"

119.82.94.132 – – [09/Feb/2010:22:36:33
 -0500] "GET /delhi/gagan-vihar.html HT
 TP/1.1" 200 24267 "http://www.google.c
 o.in/search?hl=en&source=hp&q=gagan+vi
 har+delhi&meta=&aq=0&oq=gagan+vihar&fp
 =44adc7acd595e18a" "Mozilla/5.0 (Windo
 ws; U; Windows NT 5.1; en-US; rv:1.9.1
 .7) Gecko/20091221 Firefox/3.5.7"

***********************************

From these logs we can understand that traffic is coming from google but still I fail to understand instead of writing proper keywords its reporting these numeric values. Why though the search is made on proper keywords like in line 1 if you check the search key words are hyderabad+latitude+longitude instead awstat is reporting the keyword as 12b81a5d82c30213 which is google's internal value.

When we put the same url in google it shows blank page. Please help me resolving this issue

8:32 am
February 10, 2010


Jean-Luc

Admin

posts 1125

Interesting. Seems to pick the wrong parameter because of this line in /lib/search_engines.pm .

Line 669 (version 6.95) is :

'google','(p|q|as_p|as_q)=',

I would replace it with :

'google','(q|as_q)=',

10:05 am
February 10, 2010


Deepak Gupta

Guest

Hi Jean

Thanks for your response. Where u will make changes. Is it you are going to make changes in awstats source code or we require to make these changes in some config file of awstats.

If you are making these changes in source code how will get it implemented in our version of awstats.

Deepak

11:00 am
February 10, 2010


Jean-Luc

Admin

posts 1125

Deepak,

I will report this as a bug, but I cannot tell what the AWStats developers will do.

For time being, you need to edit the file search_engines.pm . You will find it in the AWStats lib  directory.

12:04 pm
February 10, 2010


Deepak Gupta

Guest

Hi Jean

Thanks for pointing out the bug. We have made the required changes in the files will check in next compilation.

Deepak

1:40 pm
February 10, 2010


Deepak Gupta

Guest

Hi Jean

Now I have checkd the file after modifcation suggested by you but still it doesn't work out. still the no. of keyword increases. I have a doubt why it is happening only for few keyword where as other keywords are reported properly. I get below mentioned entries in awstats report. You can just note down only few keywords are there with numeric value where as other keywords are reported accurately.

Search  –  Full list Pages Hits Bandwidth Last visit
58ffa43166db4145 777 777 21.72 MB 10 Feb 2010 – 06:42
distance between cities 614 614 20.60 MB 10 Feb 2010 – 06:41
map of india 568 568 18.07 MB 10 Feb 2010 – 06:42
india map 476 476 15.16 MB 10 Feb 2010 – 06:36
maps of india 281 281 8.44 MB 10 Feb 2010 – 06:37
distance calculator 249 249 8.43 MB 10 Feb 2010 – 06:36
Failed to extract key 239 239 6.98 MB 10 Feb 2010 – 02:03
pincode search 190 190 4.26 MB 10 Feb 2010 – 06:39
pin code 175 175 3.82 MB 10 Feb 2010 – 06:40
44adc7acd595e18a 169 169 4.89 MB 10 Feb 2010 – 06:36
maps 148 148 4.05 MB 10 Feb 2010 – 06:41
distance calculator india 126 126 3.91 MB 10 Feb 2010 – 06:35
india map with cities 114 114 5.11 MB

10 Feb 2010 – 06:42

10:28 pm
February 10, 2010


Jean-Luc

Admin

posts 1125

I tried to reproduce your problem here using the log file sample you sent (17 hits). I tried with AWStats 6.9 and 6.95, but I am not able to reproduce it. There was no abnormal (hexadecimal) keywords in the report.

I guess my conclusion above (when I was talking about an AWStats bug) was not correct.

Either the problem was not present in the sample, either your copy of AWStats has been corrupted in some way.

6:14 am
February 11, 2010


Deepak Gupta

Guest

Hi Jean

I think if you check logs I have sent the value 44adc7acd595e18a is there in one line I am not able to send you exact log becuase we delete logs after compilation due to it hudge size. I may require to send some biggerlog file to you for compilation how I can send it to you as I have to send aas an attachment.

I do agree with you the solution you have given doesn't worked out as the values changed after implemtations too. Need some more analysis.

As far as copy of Awstats is corrupted, I have multiple copies installed on multiple server its same happening on all servers

You can download log files of same domain using following link.

http://sl20.in/moi-stats.tgz

I think it can give you more ideas. These log files contains value 58ffa43166db4145

The version of awstats used is

Advanced Web Statistics 6.9 (build 1.925)

8:59 am
February 11, 2010


Jean-Luc

Admin

posts 1125

I have run an AWStats (6.9) update on your moi-stats.tgz and I get 1052 valid keyphrases and no hexadecimal number. Weird.

Are you using any plugins ?

10:28 am
February 11, 2010


Deepak Gupta

Guest

Hi Jean

We are not using any plugins. I am writing the code we are using in extra section to read keywords of google

# ——————————–
# — Search Engine Optimization –
# ——————————–
# — Top Searches by specific search Eng
ines –
ExtraSectionName1="Google Searches – To
p 2000?
ExtraSectionCodeFilter1="200 304"
# — This should look like this to avoid
 mail, translate but && is not supporte
d?
# ExtraSectionCondition1="REFERER,(.*.g
oogle.*)&&REFERER,^http://([^mail.googl
e.])"
ExtraSectionCondition1="REFERER,(.*www.
google..*)"
ExtraSectionFirstColumnTitle1="Search"
ExtraSectionFirstColumnValues1="REFERER
,p=([^&]+)||REFERER,q=([^&]+)||REFERER,
as_p=([^&]+)||REFERER,as_q=([^&]+)"
ExtraSectionFirstColumnFormat1="<a href
='http://www.google.com/search?q=%s&#39; ta
rget='_blank' title='Click to execute s
earch [new window]'>%s</a>"
ExtraSectionStatTypes1=PHBL
ExtraSectionAddAverageRow1=0
ExtraSectionAddSumRow1=1
MaxNbOfExtra1=1000
MinHitExtra1=1

Check if there is any thing wrong in this code

10:57 am
February 11, 2010


Jean-Luc

Admin

posts 1125

REFERER,p=([^&]+) is the source of the problem. It selects what follows p= in Google referrers. So your extra section selects what follows fp=.

I hope this is the real solution now!

12:08 pm
February 11, 2010


Deepak Gupta

Guest

Thanks Jean

We got the solutions. What you said seems to be real root of problem. Now we have

changed the line

ExtraSectionFirstColumnValues1="REFERER
,p=([^&]+)||REFERER,q=([^&]+)||REFERER,
as_p=([^&]+)||REFERER,as_q=([^&]+)"

to

ExtraSectionFirstColumnValues1="REFERER
,q=([^&]+)||REFERER,as_q=([^&]+)"

and it seems its working. We have tested it on sample logs and impleted on

live servers. We are now waiting for result till tom when logs will get compiled again.

Thanks a lot for your help.

Reply to Post

Reply to Topic:
Problem with Extra Saction

Guest Name (Required):

Guest Email (Required):

NOTE: New Posts are subject to administrator approval before being displayed

Smileys
Confused Cool Cry Embarassed Frown Kiss Laugh Smile Surprised Wink Yell
Post New Reply

Guest URL (required)

Math Required!
What is the sum of:
9 + 12
   


About the InternetOfficer.com Forum

Forum Timezone: UTC 1

Most Users Ever Online: 302

Currently Online:
18 Guests

Currently Browsing this Topic:
1 Guest

Forum Stats:

Groups: 2
Forums: 9
Topics: 639
Posts: 2710

Membership:

There are 257 Members
There have been 304 Guests

There is 1 Admin
There is 1 Moderator

Top Posters:

cssfsu – 55
deepakgupta – 34
albert_newton – 30
cosminpana – 20
DTNMike – 19
ahtshun83 – 17

Recent New Members: raju, todd2taylor, sbdcunha, mansigill1987, ThomasDuh, ThomasKic

Administrators: Jean-Luc (1125 Posts)

Moderators: Jean-Luc (1125 Posts)