Spock


Owner of the robot : Spock

Country : USA

Robot type : search engine

Description : Spock robots crawl the web to collect informations about individuals. Spock web site speaks about indexing every human being on the planet!

Spock infrastructure uses Amazon Web Services: Amazon Simple Storage Service (S3) to store millions of profile photos and Amazon’s Elastic Compute Cloud (EC2) to ensure the appropriate scalability.

    User Agent transmitted to the visited web server :

    • Spock Crawler (http://www.spock.com/crawler)

    IP address range :

    • from 72.44.32.0 to 72.44.63.255 (amazonaws.com)
      (last visit in August 2007)

    Access control options understood by the robot :

    • robots.txt

    User Agent to use in the robots.txt file : spock

    URL for more information : http://www.spock.com/crawler