MSN Bot on Crack
I was listening to SEO Rockstars tonight and I had to laugh at the chat and comments about how bad MSN’s crawling ability is. Someone made the comment that MSN must use some kind of priority when choosing which sites to “deep crawl” and which ones to grab a few pages from. I have a site that MSN beats the crap out of day-in and day-out. It actually makes me crazy sometimes because I have a 404 handler on that site that emails me. So, when I emptied the ~6,000 emails out of that box today, I almost lost it. Then tonight the SEO Rockstar team, and people in the chatroom, started bashing MSN bot a bit. I have to agree. The best part was when I did a search for the number of pages for the “new” SEO Rockstars website…112 pages for a blog with zero content (
March 8th, 2006 at 11:38 pm
oh yeah. MSN beats the hell out of my site, too. Ironically, they beat the crap out of the www.fyreplace.com, and barely touch the politics.fyreplace.com that’s link to FROM the main blog.
They’re the reason my bandwith finally exceeded five gig in Feb (up to five gig from 1.3 gig in Jan).
It’s… annoying.
The only one that was worse was a bot by some little off-brand search engine that got stuck in my Gallery software (by, er, Gallery), and racked up over 2gb in bandwith in three days. Needless to say, I tracked down their number and //called// them real darn quick.