msnbot
Owner of the robot : Microsoft Corporation
Country : USA
Robot type : search engine
Description : Live Search (formerly MSN Search) is Microsoft search engine. MSNBot is its main crawler.
Substantial traffic has been noted from third party computers using the MSNBot user agent. The owners of these computers hope that webmasters will not block them, because of their user agent. Microsoft published Search robots in disguise, an article explaining to webmasters how to avoid confusion between the real MSNBots and the unauthorized imitations. Beware of the Chinese robot operating from addresses in the range between 202.96.51.128 and 202.96.51.255 belonging to Microsft (China) Co.Ltd which is not related to Microsoft Corporation.
According to our own data and contrary to what Microsoft states in the article, not all Microsoft hosts names end in live.com; a few still use the old Microsoft proprietary phx.gbl.
User Agent transmitted to the visited web server :
- msnbot/1.0 (+http://search.msn.com/msnbot.htm)
IP address range :
- from 65.52.0.0 to 65.55.255.255 (live.com phx.gbl msn.com)
(last visit in February 2008) - from 207.46.0.0 to 207.46.255.255 (live.com)
(last visit in August 2007)
User Agent transmitted to the visited web server :
- msnbot/1.1 (+http://search.msn.com/msnbot.htm)
IP address range :
- from 65.52.0.0 to 65.55.255.255 (live.com phx.gbl msn.com)
(last visit in May 2008)
Access control options understood by the robot :
- robots.txt
- META NAME=”robots”
- rel=”nofollow”
User Agent to use in the robots.txt file : msnbot
URL for more information :
Our pages about other Microsoft robots :
September 16th, 2007 at 4:49 am
I recently implemented a PHP script to exclude fake robots from browsing my site. The script attempts to find the host of the originating IP address of user agents identifying themselves as Googlebot, MSNbot, or SLurp, then compares it to the hostnames published by Google, MSN, and Yahoo. I was confused by the phx.gbl domain name until I found several reports of this domain in other areas. Until this issue is resolved at MSN, I have no option other than allowing the phx.gbl domain, and skipping the reverse lookup, as it would obviously fail.
I did notice in my log files that I have had fake MSNbot hits from the 202.96.51.128 – 202.96.51.255 range of Microsft (China) Co.Ltd, as well as 219.141.128.0 – 219.143.255.255, which belongs to CHINANET beijing province network.