InternetOfficer.com

Do you want to know everything about Googlebot, Yahoo! Slurp or MSNBot ? Your site has been visited by a mysterious spider ? A visitor with an unusual User Agent searches all your site ? Will this robot bring you new visitors or is it just overloading your bandwidth ?

If you are in charge of Search Engine Optimisation or if you are a webmaster, you need to know where crawlers (spiders) visiting your site come from. The Web Robots Encyclopedia has the answers.

 
• A
  AbiLogic   Accoona   AdSense   aipbot   Alexa   Almaden   AOL France   appie   Ask Jeeves   ASPseek  
   
• B
  Baidu   baiduspider   BecomeBot   Bloglines   BlogPulse   Boitho   btbot   Burf.com  
   
• C
  Camcrawler   Camdiscover   Cazoodle   CazoodleBot   Cerberian   cfetch   Charlotte   CheckWeb   Combine   Cosmix   cuill.com  
   
• D
  DataCha0s   DataparkSearch   dir.com   DTS Agent  
   
• E
  e-SocietyRobot   Enfin   Entireweb   Everbee   exabot   Exabot-Images   Exalead  
   
• F
  Fast Search   Filangy   Findexa   FindLinks   FURL  
   
• G
  genieBot   Gigablast   gigabot   Girafa   Google   Google Image   Google-Sitemaps   Googlebot   Googlebot-Image   Gossamer Links   Grub  
   
• H
  Healthline   HenryTheMiragoRobot   Heritrix  
   
• I
  ia_archiver   IBM WebFountain   ichiro   Internet Archive   Ipselon   ipselonbot   IRL crawler   IRLbot  
   
• J
  Java   Jeeves  
   
• K
  Kavam   Kosmix   Krugle  
   
• L
  LapozzBot   Larbin   Link Valet   LinkChecker   LinksManager   LinkWalker   Live   Local.com   LookSmart   LOOP Improvements   Lycos Spider  
   
• M
  Majestic-12   Mediapartners   MetaWeb Crawler   Microsoft Antispam Bot   Mirago   Misterbot   MJ12bot   MMCrawler   mnoGoSearch   MojeekBot   MSN   msnbot   MSNPTC   MSRA   MSRBot   Mylinea  
   
• N
  NaverBot   NetResearchServer   Nexen   NextGenSearchBot   NG   NimbleCrawler   Norbert   noXtrum   Nusearch   Nutch  
   
• O
  Omni-Explorer   OnetSzukaj   oo-x   Orangeslicer   Orbiter   Oxado  
   
• P
  PeerFactor   PHP version tracker   Picsearch   Plantynet   pmafind   pompos   Poodle   psbot   psycheclone  
   
• Q
  QweeryBot  
   
• R
  Rambler   RedKernel   Rufus Web Miner  
   
• S
  SAGOOL   SandCrawler   SBIder   Scirus   Searchmee   SearchSerf   Secure Computing   seekbot   Seekport   sensis.com.au   Shim-Crawler   ShopWiki   Singingfish   SiteSell.com   Slurp   snafu   Snapbot   SoGou   SOHU.com   Speedy   Spock   Sproose   SquidClamAV   StackRambler   SurveyBot   SygolBot  
   
• T
  Teoma   Tilman   TLink   TMCrawler   TurnitinBot   Twiceler  
   
• U
  Ulysseek  
   
• V
  Vagabondo   Voila   Voyager  
   
• W
  Wadaino   Walhello   WebFilter   WebFountain   Whois Source   Wikia   WinkBot   WiseNut   WorldIndexer   wwwster   WyseGuys  
   
• X
  Xenu  
   
• Y
  Y!J-BSC   YaCy   Yahoo! Slurp   Yahoo-Blogs   Yahoo-MMCrawler   Yeti   Yodao   YodaoBot  
   
• Z
  Zeusbot   ZoomInfo   ZyBorg  

The Web Robots Encyclopedia is designed and maintained by InternetOfficer.