IRL crawler


Owner of the robot : Texas A&M Internet Research Lab

Country : USA

Robot type : search engine

Description : Research project sponsored by the National Science Foundation that investigates algorithms for mapping the topology of the Internet.

User Agent transmitted to the visited web server :

  • IRLbot/2.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler)

 


 
IP address range : from 128.194.0.0 to 128.194.255.255 (tamu.edu)

URL for more information : http://irl.cs.tamu.edu/crawler/

Access control options understood by the robot :

  • robots.txt
  • META NAME=”robots”

User Agent to use in the robots.txt file : IRLbot

 
Last visit of this robot logged in July 2006.
Other informations updated on February 3, 2006.