InternetOfficer.com

32.97.110.142 – - [15/Oct/2008:08:59:17 00000] “GET /software/brandcatalog/portal/opal/details?NavCode=1TW101003 HTTP/1.1″ 200 7152 “-” "swish-e spider 2.2 http://swish-e.org/" “-” 0.180 TCP_MISS_PRIVATE_ON_STOP_LIST/200 7616 GET http://www-01.xyz.com/software/brandcatalog/portal/opal/details?NavCode=1TW101003 DIRECT/9.57.57.42 “text/html; charset=UTF-8″ – )”

Above is an example of a record in my log file with the spider information.. This is messing up my extra section as all such hits from the spider shows up there as well..

Any way I can exclude this and/or other spiders from my extra sections count which is below..

ExtraSectionName1=”navCode Details (Top 500 terms)”
ExtraSectionCodeFilter1=”200 304″
ExtraSectionCondition1=”URL,\\/software\\
/brandcatalog\\/portal\\/soa\\/details||UR
L,\\/software\\/brandcatalog\\/myportal\\/so
a\\/details”
ExtraSectionFirstColumnTitle1=”navCode”
ExtraSectionFirstColumnValues1=”QUERY_S
TRING,catalog.label=([^&]+)”
ExtraSectionFirstColumnFormat1=”%s”
ExtraSectionStatTypes1=PL
ExtraSectionAddAverageRow1=0
ExtraSectionAddSumRow1=1
MaxNbOfExtra1=500
MinHitExtra1=1\\
ExtraTrackedRowsLimit=1200

I tried to omit the spider by adding this line in the conf file, but it didnt help..

SkipUserAgents=”swish-e spider 2.2 http://swish-e.org/“

Plz help as i am stuck over here..

No it didnt work.. Still the extra section shows the hits by the spider..  It seems that there is no way to exclude the spider hits in the extra section.. If u ve any more ideas, do let me know..

thanks !