MSN Crawlers Pawned

After seeing the way MSN crawled my last post, I just realized why Microsoft could never do good in Search Engine Market :)

I wonder why MSN would crawler same page from two different machines. I wonder if a single page can be divided further for crawling. Checkout the screenshots below :)

Why Google Wins

A few minutes later. Three Microsoft machines were crawling the same page 😛

MSN Crawler Pawned

I have conformed using ARIN that all these IPs belong to Microsoft 😀

 

One thought on “MSN Crawlers Pawned

Comments are closed.