Re: Majestic 12 bot

#1

On Sun, 09 Oct 2005 11:09:15 GMT, Darren Tipton <news (AT) appletippy (DOT) org>
wrote:

Quote:

Have any of you been following the majestic 12 project? It's bot has been
deep crawling one of my sites consistently (gets good ranking on their SE as
well).

Haven't seen anything in the news about, spotted it in the site logs.

http://majestic12.co.uk

It's never hit my site
Here are the ones that do in order of hits (busiest first)

1 Hotbot-Lycos Robot
2 Googlebot
3 psbot
4 Direct Hit/Ask/Askjeeves Robot
5 Alexa Robot
6 www.baidu.com
7 libwww-perl
8 appie
9 MSIE Crawler
10 almaden.ibm
11 Perl LWP: Collective UserAgent module
12 Xenu Link Sleuth
13 WGet
14 www.merl.com

--
I see dead people.
But I don't see David's messages

http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----

#2

"Paul" <webmaster (AT) houstoncrafts (DOT) com> wrote

Quote:

On Sun, 09 Oct 2005 11:09:15 GMT, Darren Tipton <news (AT) appletippy (DOT) org
wrote:

Have any of you been following the majestic 12 project? It's bot has been
deep crawling one of my sites consistently (gets good ranking on their SE
as
well).

Haven't seen anything in the news about, spotted it in the site logs.

http://majestic12.co.uk

It's never hit my site
Here are the ones that do in order of hits (busiest first)

snip

Quote:

--
I see dead people.
But I don't see David's messages

To scary.....do ya have to hold a séance? You can see Dave's messages if
Bill replies to them. :-)

My Modlogan logs show these as robot visits.
1. Yahoo! Slurp
2. RedCarpet/1.2 ----- This one states it will be up late 2005(better
hurry huh) when I visited the site http://redcarpet-inc.com/. Don't know why
they have the bots this busy! Good thing I have plenty of bandwidth.:-)
3. msnbot/1.0
4. Mediapartners-Google/2.1
5. Googlebot/2.1
6. BecomeBot/2.3
7. ShopWiki/1.0
8. Lycos_Spider_(modspider)
9. psbot/0.1
10. Ask Jeeves/Teoma
11. Unknown
12. Gigabot/2.0
13. MSIE 6.0
14. ConveraMultiMediaCrawler/0.1
15. compatible Mozilla/2.0
16. ia_archiver
17. PrivacyFinder/1.1
18. larbin_2.6.3 (wgao (AT) genieknows (DOT) com)
19. Sensis Web Crawler
20. envolk[ITS]spider/1.6
21. StackRambler/2.0 (MSIE incompatible)
22. ichiro/1.0 (ichiro (AT) nttr (DOT) co.jp)
21. SygolBot
22. Zeus 97728 Webster Pro V2.9 Win32 ---- Someone scraping here huh?:-)
23. http://www.almaden.ibm.com/cs/crawler
24. Scooter/3.3
25. MSIE 5.5
26. Python-urllib/1.16
27. Scrubby/2.2

On another note here are the top search engines that are used to come to my
site:

1. 47.29% "Google" this isn't counting the cached views used. Like
http://64.233.187.104/search?q=cache...yourself&hl=en
this has 12 hits. Wow.:-)
2. 22.88% "MSN"
3. 10.79% "Yahoo"

At any rate Google takes the lead.:-)

Stacey

#3

On Sun, 9 Oct 2005 13:59:10 +0200, "Stacey"
<Remove-the-Y-stacey (AT) staceyssimplestuff (DOT) com> wrote:

Quote:

"Paul" <webmaster (AT) houstoncrafts (DOT) com> wrote in message
news:vgvhk11b0t92c4t49cvkb7abdfrhdim7i5 (AT) 4ax (DOT) com...
On Sun, 09 Oct 2005 11:09:15 GMT, Darren Tipton <news (AT) appletippy (DOT) org
wrote:

Have any of you been following the majestic 12 project? It's bot has been
deep crawling one of my sites consistently (gets good ranking on their SE
as
well).

Haven't seen anything in the news about, spotted it in the site logs.

http://majestic12.co.uk

It's never hit my site
Here are the ones that do in order of hits (busiest first)

snip

--
I see dead people.
But I don't see David's messages

To scary.....do ya have to hold a séance? You can see Dave's messages if
Bill replies to them. :-)

I shall cease immediament!

BB
--
www.kruse.co.uk/ seo (AT) kruse (DOT) demon.co.uk
Elvis does my SEO

#4

On Sun, 9 Oct 2005 13:59:10 +0200, "Stacey"
<Remove-the-Y-stacey (AT) staceyssimplestuff (DOT) com> wrote:

Quote:

I see dead people.
But I don't see David's messages

To scary.....do ya have to hold a séance? You can see Dave's messages if
Bill replies to them. :-)

That's quite funny since I don't see Bill's posts.

Bill, Bill reply to this so what's his name (forgot his name) sees it.

LOL

David
--
Free Search Engine Optimization Tutorial
http://www.seo-gold.com/tutorial/

#5

On Sun, 9 Oct 2005 13:59:10 +0200, "Stacey"
<Remove-the-Y-stacey (AT) staceyssimplestuff (DOT) com> wrote:

Quote:

"Paul" <webmaster (AT) houstoncrafts (DOT) com> wrote in message
news:vgvhk11b0t92c4t49cvkb7abdfrhdim7i5 (AT) 4ax (DOT) com...
On Sun, 09 Oct 2005 11:09:15 GMT, Darren Tipton <news (AT) appletippy (DOT) org
wrote:

Have any of you been following the majestic 12 project? It's bot has been
deep crawling one of my sites consistently (gets good ranking on their SE
as
well).

Haven't seen anything in the news about, spotted it in the site logs.

http://majestic12.co.uk

It's never hit my site
Here are the ones that do in order of hits (busiest first)

snip

--
I see dead people.
But I don't see David's messages

To scary.....do ya have to hold a sÃ©ance?

Oh no, I just open the lid of the Urn and can see my father.

Quote:

You can see Dave's messages if Bill replies to them. :-)

Yes, but he has never anything valid to say. I don't have much time
for inbreds.

Quote:

My Modlogan logs show these as robot visits.
1. Yahoo! Slurp
2. RedCarpet/1.2 ----- This one states it will be up late 2005(better
hurry huh) when I visited the site http://redcarpet-inc.com/. Don't know why
they have the bots this busy! Good thing I have plenty of bandwidth.:-)
3. msnbot/1.0
4. Mediapartners-Google/2.1
5. Googlebot/2.1
6. BecomeBot/2.3
7. ShopWiki/1.0
8. Lycos_Spider_(modspider)
9. psbot/0.1
10. Ask Jeeves/Teoma
11. Unknown
12. Gigabot/2.0
13. MSIE 6.0
14. ConveraMultiMediaCrawler/0.1
15. compatible Mozilla/2.0
16. ia_archiver
17. PrivacyFinder/1.1
18. larbin_2.6.3 (wgao (AT) genieknows (DOT) com)
19. Sensis Web Crawler
20. envolk[ITS]spider/1.6
21. StackRambler/2.0 (MSIE incompatible)
22. ichiro/1.0 (ichiro (AT) nttr (DOT) co.jp)
21. SygolBot
22. Zeus 97728 Webster Pro V2.9 Win32 ---- Someone scraping here huh?:-)
23. http://www.almaden.ibm.com/cs/crawler
24. Scooter/3.3
25. MSIE 5.5
26. Python-urllib/1.16
27. Scrubby/2.2

That's some list you have there

I have plenty of b/w too, but it is now using 2gb a day and thankfully
don't have all those bots visiting.

Quote:

On another note here are the top search engines that are used to come to my
site:

1. 47.29% "Google" this isn't counting the cached views used. Like
http://64.233.187.104/search?q=cache...yourself&hl=en
this has 12 hits. Wow.:-)
2. 22.88% "MSN"
3. 10.79% "Yahoo"

At any rate Google takes the lead.:-)

Stacey

Take care
Paul

--
I see dead people.
But I don't see David's messages

http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----

#6

"David" <seodave (AT) search-engine-optimization-services (DOT) co.uk> wrote

Quote:

On Sun, 9 Oct 2005 13:59:10 +0200, "Stacey"
Remove-the-Y-stacey (AT) staceyssimplestuff (DOT) com> wrote:

I see dead people.
But I don't see David's messages

To scary.....do ya have to hold a séance? You can see Dave's messages if
Bill replies to them. :-)

That's quite funny since I don't see Bill's posts.

Bill, Bill reply to this so what's his name (forgot his name) sees it.

LOL

:-)

#7

On Sun, 9 Oct 2005 16:53:39 +0200, "Stacey"
<Remove-the-Y-stacey (AT) staceyssimplestuff (DOT) com> wrote:

Quote:

"David" <seodave (AT) search-engine-optimization-services (DOT) co.uk> wrote in message
news:5f5ik1ta3gtebmguqdhrip1jmtsetdf8nr (AT) 4ax (DOT) com...
On Sun, 9 Oct 2005 13:59:10 +0200, "Stacey"
Remove-the-Y-stacey (AT) staceyssimplestuff (DOT) com> wrote:

I see dead people.
But I don't see David's messages

To scary.....do ya have to hold a sÃ©ance? You can see Dave's messages if
Bill replies to them. :-)

That's quite funny since I don't see Bill's posts.

Bill, Bill reply to this so what's his name (forgot his name) sees it.

LOL

:-)

Yep Stacey, I see his pathetic reply (as ususal)

Bit early for him to be online as he is normally still in bed with his
mother at this time of day.
May be his brother wanted to slip her one, so he got up and posted
instead

plh
Paul

--
Wank a man off, and he will enjoy himself for a day. Teach a man how to wank, and he'll be a wanker for the rest of his life.

http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----

#8

On Sun, 09 Oct 2005 08:46:04 -0500, Paul <webmaster (AT) houstoncrafts (DOT) com>
wrote:

Quote:

To scary.....do ya have to hold a sÃ©ance?

Oh no, I just open the lid of the Urn and can see my father.

You can see Dave's messages if Bill replies to them. :-)

Yes, but he has never anything valid to say. I don't have much time
for inbreds.

Hey, only I'm allowed to insult Bill that way!!

David
--
Free Search Engine Optimization Tutorial
http://www.seo-gold.com/tutorial/

#9

"Paul" <webmaster (AT) houstoncrafts (DOT) com> wrote

<snip>

Quote:

That's some list you have there

I have plenty of b/w too, but it is now using 2gb a day and thankfully
don't have all those bots visiting.

Here is the way I look at it. The more bots you have to visit makes your
site more accessible in the web. Even those little small ones. There is
probably a couple that doesn't need to be there, but hey like I said more
accessible IMO. I mean scrubtheweb isn't that big but it is just another
listing that your site could be on.

2g's a day, that is a lot! You must have a lot of traffic and large files.

Stacey

#10

On Mon, 10 Oct 2005 00:15:56 +0200, "Stacey"
<stacey (AT) staceyssimplestuff (DOT) com> wrote:

Quote:

"Paul" <webmaster (AT) houstoncrafts (DOT) com> wrote in message
news:5k7ik11bqg69nebssh0t135ojsslnb8c32 (AT) 4ax (DOT) com...wrote:

snip

That's some list you have there

I have plenty of b/w too, but it is now using 2gb a day and thankfully
don't have all those bots visiting.

Here is the way I look at it. The more bots you have to visit makes your
site more accessible in the web. Even those little small ones. There is
probably a couple that doesn't need to be there, but hey like I said more
accessible IMO. I mean scrubtheweb isn't that big but it is just another
listing that your site could be on.

2g's a day, that is a lot! You must have a lot of traffic and large files.

It's a large site, yes.
I have almost 2000 a day unique visitors now and broke the 1.5 million
pages read on friday just gone.

Some visitors stay for a few minutes, others for an hour or so. The
longest was on there for over 7.5 hours.

The files are not that large. The average wallpaper size.

There are over 2000 images on the site and I have enough to add
another 3000 + more. But I cannot afford the extra charges in b/w
otherwise I would.

Quote:

Stacey

Take care
Paul

--
Wank a man off, and he will enjoy himself for a day. Teach a man how to wank, and he'll be a wanker for the rest of his life.

http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----