Archive for June, 2007

ShopWiki

Monday, June 11th, 2007

Owner of the robot : ShopWiki Corp.
Country : USA
Robot type : search engine
Description : A shopping search engine.
User Agent transmitted to the visited web server :

ShopWiki/1.0 ( +http://www.shopwiki.com/wiki/Help:Bot)

 

 
IP address range : from 38.0.0.0 to 38.255.255.255 ()
URL for more information : http://www.shopwiki.com/wiki/Help:Bot
Access control options understood by the robot :

robots.txt
META NAME=”robots”

User Agent to use in the [...]

Shim-Crawler

Monday, June 11th, 2007

Owner of the robot : University of Tokyo
Country : Japan
Robot type : search engine
Description : Search engine of the Chikayama-Taura laboratory of the University of Tokyo. This search engine is only used for performing researches and not for any business activity.
User Agent transmitted to the visited web server :

Shim-Crawler(Mozilla-compatible; http://www.logos.ic.i.u-tokyo.ac.jp/crawler/; crawl@logos.ic.i.u-tokyo.ac.jp)

 

 
IP address range : from [...]

Sensis

Monday, June 11th, 2007

Owner of the robot : Telstra
Country : Australia
Robot type : search engine
Description : sensis.com.au advertises itself as the search engine for Australians. It offers unique search capabilities in Trading Post OnLine and Australian yellow and white pages.

User Agent transmitted to the visited web server :

Sensis Web Crawler (search_comments\\at\\sensis\\dot\\com\\dot\\au)

IP address range :

from 70.42.51.0 to 70.42.51.127 ()(last [...]

Seekport

Monday, June 11th, 2007

Owner of the robot : Seekport Internet Technologies GmbH
Country : Germany
Robot type : search engine
Description : Seekport is a European provider of “white-label” web-search services to portals, ISP’s and other corporations who desire to integrate a search functionality in their own offerings. Seekport headquarters are located in Germany, with offices in the UK, France, Italy [...]

Secure Computing

Monday, June 11th, 2007

Owner of the robot : Secure Computing Corporation
Country : USA
Robot type : search engine
Description : The robot of the Web Filtering Tools group at Secure Computing Corporation spiders sites, attempting to locate virus-infected web content.
User Agent transmitted to the visited web server :

virus_detector (virus_harvester@securecomputing.com)
virus_detector virus_harvester@securecomputing.com

 

 
IP address range : from 210.169.110.64 to 210.169.110.95 (securecomputing.com)
URL for more [...]

SearchSerf

Monday, June 11th, 2007

Owner of the robot : Paul Le Roux
Country : South Africa
Robot type : search engine
Description : Search engine under development.
User Agent transmitted to the visited web server :

Mozilla/3.0 (compatible; ScollSpider; http://search.Market-UK.com)

 

 
IP address range : from 82.43.32.0 to 82.44.105.255 (blueyonder.co.uk)
URL for more information :
Access control options understood by the robot :

User Agent to use [...]

Searchmee

Monday, June 11th, 2007

Owner of the robot : Fred Stiening
Country : USA
Robot type : search engine
Description : Search engine under development.
User Agent transmitted to the visited web server :

Searchmee! Spider v0.98a

 

 
IP address range : from 64.202.96.0 to 64.202.127.255 (findanisp.com)
URL for more information :
Access control options understood by the robot :

User Agent to use in the robots.txt [...]

Scirus

Monday, June 11th, 2007

Owner of the robot : Elsevier B.V.
Country : Netherlands
Robot type : search engine
Description : Science-specific search engine.
User Agent transmitted to the visited web server :

FAST Enterprise Crawler 6 / Scirus scirus-crawler@fast.no; http://www.scirus.com/srsapp/contactus/

 

 
IP address range : from 66.151.181.0 to 66.151.181.255 (fastsearch.net)
URL for more information : http://www.scirus.com/srsapp/aboutus/
Access control options understood by the robot :

robots.txt

User Agent to [...]

SandCrawler

Monday, June 11th, 2007

Owner of the robot : Microsoft Corporation
Country : USA
Robot type : unknown
Description : No information available.
User Agent transmitted to the visited web server :

SandCrawler - Compatibility Testing

 

 
IP address range : from 131.107.0.0 to 131.107.255.255 (microsoft.com)
URL for more information :
Access control options understood by the robot :

User Agent to use in the robots.txt [...]

SAGOOL

Monday, June 11th, 2007

Owner of the robot : SAGOOL
Country : Japan
Robot type : search engine
Description : Experimental Japanese search engine.
User Agent transmitted to the visited web server :

MaSagool/1.0 (MaSagool; http://sagool.jp/; masagool@sagool.jp)

 

 
IP address range : from to ()
URL for more information : http://sagool.jp/about.html
Access control options understood by the robot :

User Agent to use in the robots.txt [...]