Re: Googlebot becoming obnoxious

__/ [Ignoramus25589] on Friday 14 October 2005 05:31 \__

It crawls my websites algebra.com and cooldictionary.com way too
fast.

Today, for instance, algebra.com received 67,000 accesses from
googlebot.

They also misbehave and do not honor Crawl-delay setting.

I am not going to ban googlebot from crawling my site, but I am
becoming somewhat apprehensive of them.

I want to limit their accesses to, say, no more than every 5
seconds. How can I do it, realistically speaking.

i

{
* Nice to have you back in this NG

* Mikkel, I am aware of the subject line which will be bound to my name in
SEPR's... ouch!
}

Someone in this newsgroup asked a similar question a month or two back. It
is not quite possible to limit the demands that reach from Google if I
recall correctly. That was the conclusion reached, I think. My recollection
tells me that the best move would be to contact Google saying that your
dedicated server is knocked on too hard.

(since I bound my name to the subject line, I might as well go all the way)

001 650 330 0100

Call them to get an E-mail address and then be in touch with a real person
promptly.

Hope it helps,

Roy

--
Roy S. Schestowitz | Have you hugged your penguin today?
http://Schestowitz.com | SuSE Linux | PGP-Key: 74572E8E
6:10am up 49 days 18:24, 4 users, load average: 0.82, 0.84, 0.73
http://iuron.com - next generation of search paradigms

Roy Schestowitz wrote:

* Mikkel, I am aware of the subject line which will be bound to my name in
SEPR's... ouch!

Hehe... You are one of the few people that I know on usenet who bothers
to change the subjectline for subthreads unrelated to the original post;
some people actually argue against doing it.

Now I know why you do that :-)

The title of this is intended to give an example of how to rephrase
subject lines in a career-friendly way :-D

Mikkel

--
Guatemala travel, Spanish studies, volunteering:
http://lakjer.dk/mikkel/spanish.shtml