IRL crawler

Owner of the robot : Texas A&M Internet Research Lab

Country : USA

Robot type : search engine

Description : Research project sponsored by the National Science Foundation that investigates algorithms for mapping the topology of the Internet.

User Agent transmitted to the visited web server :

  • IRLbot/2.0 (compatible; MSIE 6.0;


IP address range : from to (

URL for more information :

Access control options understood by the robot :

  • robots.txt
  • META NAME=”robots”

User Agent to use in the robots.txt file : IRLbot

Last visit of this robot logged in July 2006.
Other informations updated on February 3, 2006.