Issue with MSNBot

I'm having a recurring issue with MSNBot.

It keeps requesting pages that do not exist within our site. The site
uses a CMS and pages are retreived by passing a page id in the
querystring to the default.aspx page eg: default.aspx?page=8.

MSNBot keeps making requests to the site with invalid query values like
default.aspx?12 ( no page= included ) or it requests page values that
no longer exist. I've noticed that their cached version of our site
contains a link to the non-existing pages but should it not now be
spidering off of the current site not the cached one.

I've also added a robots.txt file that blocks MSNBot's access to all
except one file contained within a folder, this file contains one link
to the root with default.aspx?page=0 in the hope that MSNBot will come
here first and then get redirected but this does not appear to work
either.

Can anyone shed some light on why MSNBot might be behaving this way ie:
Sending requests for non existing pages.
Any feedback would be greatly appreciated.

dev (AT) thehexx (DOT) com wrote:

Can anyone shed some light on why MSNBot might be behaving this way ie:
Sending requests for non existing pages.

Yes, it is written by Microsoft so it is buggy. Lots of folks are having
problem with MSNBot going on a site feeding frenzy at the mo and doing
all sorts of weird stuff.

<dev (AT) thehexx (DOT) com> wrote

I'm having a recurring issue with MSNBot.

I've also added a robots.txt file that blocks MSNBot's access to all
except one file contained within a folder, this file contains one link
to the root with default.aspx?page=0 in the hope that MSNBot will come
here first and then get redirected but this does not appear to work
either.

You could try using the msnbot meta tag?

http://www.seoconsultants.com/meta-t...ots/msnbot.asp

Rick