Hello MSN, Calling MSN
Seriously, I like the MSN Search team’s efforts in making a voracious crawler named MSNbot. But, here is the thing. When a page comes up 404, say for 30 days in a row…maybe it is time to stop asking for it? You see, there was a little “test” we may have been doing with some “questionable” pages a while back. I have since removed the pages from the site in question, yet MSNbot has not been able to figure out that 404 means “no more.” Sorry, I just thought it was too catchy to pass up. I am sure paying attention to these things after a while would save the boys and girls at MSNSearch some computing power as well as bandwidth. Just a thought.
