bandwidth google bot

Hello there

I would like to know if this following, in my awstats information is
normal:

Robots/Spiders visitors (Top 25)

Googlebot 13742 608.88 MB 20 Apr 2006 - 04:28

MSNBot 1380 68.62 MB 20 Apr 2006 - 01:32
Inktomi Slurp 447 19.95 MB 20 Apr 2006 - 04:11
Unknown robot (identified by 'spider') 374 15.71 MB 20 Apr 2006 - 01:55
Unknown robot (identified by 'crawl') 65 380.33 KB 19 Apr 2006 - 06:14
AskJeeves 11 215.16 KB 16 Apr 2006 - 03:58
Voyager 3 283.75 KB 20 Apr 2006 - 05:06
WISENutbot 2 29.29 KB 05 Apr 2006 - 16:36
Unknown robot (identified by 'robot') 2 59.22 KB 06 Apr 2006 - 18:28
Voila 2 57.23 KB 14 Apr 2006 - 17:05
Netcraft 1 0 06 Apr 2006 - 03:08


Google needs to "consume" 608.88 MB ?

xptomarketing (AT) gmail (DOT) com wrote:

Hello there

I would like to know if this following, in my awstats information is
normal:

Robots/Spiders visitors (Top 25)

Googlebot 13742 608.88 MB 20 Apr 2006 - 04:28

MSNBot 1380 68.62 MB 20 Apr 2006 - 01:32
Inktomi Slurp 447 19.95 MB 20 Apr 2006 - 04:11
Unknown robot (identified by 'spider') 374 15.71 MB 20 Apr 2006 - 01:55
Unknown robot (identified by 'crawl') 65 380.33 KB 19 Apr 2006 - 06:14
AskJeeves 11 215.16 KB 16 Apr 2006 - 03:58
Voyager 3 283.75 KB 20 Apr 2006 - 05:06
WISENutbot 2 29.29 KB 05 Apr 2006 - 16:36
Unknown robot (identified by 'robot') 2 59.22 KB 06 Apr 2006 - 18:28
Voila 2 57.23 KB 14 Apr 2006 - 17:05
Netcraft 1 0 06 Apr 2006 - 03:08


Google needs to "consume" 608.88 MB ?

It depends.

Most of all it depends on size of your site, how many pages are there?

Then, it depends on if you are hit by the old bots or by the new one?
Probably the old bots, coming from different IP's.

If they switch to the new one, your trafic will go down.



--
Website design:
http://vision2form.nl/websitedesign/
Search engine optimization:
http://vision2form.nl/websitedesign/being-found.html

xptomarket... (AT) gmail (DOT) com schreef:

Hello there

I would like to know if this following, in my awstats information is
normal:

Robots/Spiders visitors (Top 25)

Googlebot 13742 608.88 MB 20 Apr 2006 - 04:28

MSNBot 1380 68.62 MB 20 Apr 2006 - 01:32
Inktomi Slurp 447 19.95 MB 20 Apr 2006 - 04:11
Unknown robot (identified by 'spider') 374 15.71 MB 20 Apr 2006 - 01:55
Unknown robot (identified by 'crawl') 65 380.33 KB 19 Apr 2006 - 06:14
AskJeeves 11 215.16 KB 16 Apr 2006 - 03:58
Voyager 3 283.75 KB 20 Apr 2006 - 05:06
WISENutbot 2 29.29 KB 05 Apr 2006 - 16:36
Unknown robot (identified by 'robot') 2 59.22 KB 06 Apr 2006 - 18:28
Voila 2 57.23 KB 14 Apr 2006 - 17:05
Netcraft 1 0 06 Apr 2006 - 03:08


Google needs to "consume" 608.88 MB ?

Do you have a forum? Is the bot hitting here the most?

PS:There is not only one mozillabot from Google, there are a lot of new
bots(ip's).

xptomarketing (AT) gmail (DOT) com wrote:

Hello there

I would like to know if this following, in my awstats information is
normal:

Robots/Spiders visitors (Top 25)

Googlebot 13742 608.88 MB 20 Apr 2006 - 04:28

MSNBot 1380 68.62 MB 20 Apr 2006 - 01:32
Inktomi Slurp 447 19.95 MB 20 Apr 2006 - 04:11
Unknown robot (identified by 'spider') 374 15.71 MB 20 Apr
2006 - 01:55 Unknown robot (identified by 'crawl') 65 380.33
KB 19 Apr 2006 - 06:14 AskJeeves 11 215.16 KB 16 Apr
2006 - 03:58 Voyager 3 283.75 KB 20 Apr 2006 - 05:06
WISENutbot 2 29.29 KB 05 Apr 2006 - 16:36
Unknown robot (identified by 'robot') 2 59.22 KB 06 Apr
2006 - 18:28 Voila 2 57.23 KB 14 Apr 2006 - 17:05
Netcraft 1 0 06 Apr 2006 - 03:08


Google needs to "consume" 608.88 MB ?

do you have images?
are you sure that Googlebot is *just* Googlebot?

You might get a better idea when you download your access log and use:
http://johnbokma.com/perl/googlebot-statistics.html

which shows which files and how often.

Using sitemap might help you.


--
John Experienced (web) developer: http://castleamber.com/

Perl RSS Builder: http://johnbokma.com/perl/rss-web-feed-builder.html

__/ [ John Bokma ] on Thursday 20 April 2006 22:36 \__

xptomarketing (AT) gmail (DOT) com wrote:

Hello there

I would like to know if this following, in my awstats information is
normal:

Robots/Spiders visitors (Top 25)

Googlebot 13742 608.88 MB 20 Apr 2006 - 04:28

MSNBot 1380 68.62 MB 20 Apr 2006 - 01:32
Inktomi Slurp 447 19.95 MB 20 Apr 2006 - 04:11
Unknown robot (identified by 'spider') 374 15.71 MB 20 Apr
2006 - 01:55 Unknown robot (identified by 'crawl') 65 380.33
KB 19 Apr 2006 - 06:14 AskJeeves 11 215.16 KB 16 Apr
2006 - 03:58 Voyager 3 283.75 KB 20 Apr 2006 - 05:06
WISENutbot 2 29.29 KB 05 Apr 2006 - 16:36
Unknown robot (identified by 'robot') 2 59.22 KB 06 Apr
2006 - 18:28 Voila 2 57.23 KB 14 Apr 2006 - 17:05
Netcraft 1 0 06 Apr 2006 - 03:08


Google needs to "consume" 608.88 MB ?

do you have images?

This aggregate includes images. But your point is a very valid one. Google
gets hungry sometimes and sometimes there is abatement. It's quite
unpredictable. A sheer example of that is Alexa (the Web Archive). This
month, MSN seems to be the 'bad guy'.

MSNBot 40296+704 1.01 GB 21 Apr 2006 - 07:53
Googlebot 34236+24 423.33 MB 21 Apr 2006 - 07:39
Inktomi Slurp 30905+2648 512.23 MB 21 Apr 2006 - 07:59

What do I get for the 1 GB of largely-on-the-fly-generated content?

MSN 58 128

At some point, I thought about kicking MSN out...

http://schestowitz.com/Weblog/archiv...versus-msnbot/

are you sure that Googlebot is *just* Googlebot?

You might get a better idea when you download your access log and use:
http://johnbokma.com/perl/googlebot-statistics.html

which shows which files and how often.

Using sitemap might help you.

I agree on that account!

Best wishes,

Roy

--
Roy S. Schestowitz | "Have you compiled your kernel today?"
http://Schestowitz.com | SuSE Linux ¦ PGP-Key: 0x74572E8E
9:30am up 43 days 23:13, 7 users, load average: 0.44, 0.58, 0.48
http://iuron.com - help build a non-profit search engine