 | |
Google speed of spidering and indexing
Search Engine Optimization Discussion about SEO/Search Engine Optimization (alt.internet.search-engines)
 |
| | Thread Tools | Display Modes |
#1
 
| |||
| |||
|
Google speed of spidering and indexing -
06-12-2004
, 03:27 PM
#2
| |||
| |||
|
|
Hi, Here is something interesting for you all to watch. Just registered a new domain http://www.charles-dickens.org/. It was registered around 5-30pm (GMT) today and had content by 6-00pm. It's 8-00pm and currently there are approx. 600 pages online. |
BB
Quote:
|
The home page of this domain is currently linked to from the 600 pages and 27,000 other pages. Right now the only visitors to the site is me and the Google adwords bot (Mediapartners-Google/2.1). I'll probably add few thousand more pages tonight (if I get the time). What I'm going to be tracking is how long it takes for Google to find the site and for it to show in the index via a allinurl search- allinurl:charles-dickens.org/ That's funny, the home page is already showing for this search, but with no cache. Interesting. So google has managed to add the home page by 7-30pm even though the Google spider hasn't visited! I've got the toolbar installed and google ads (the latter isn't supposed to be a help). I don't submit sites to google, I rely on links for indexing. So either it's been added because I use the toolbar or because Google has spidered one (or more) of the 27,000 pages linking to it and added the home page without spidering it. Either way that was fast. Will try to check the sites allinurl: search every hour and would be interested if anyone else checks when they see more pages listed, especially with a cache since that means a page has been fully spidered and indexed. I've never checked a pages indexing speed in this much detail, best I've seen is a round 12hrs, but could of been much earlier since I wasn't checking on a regular basis. David |
#3
| |||
| |||
|
|
Hi, Here is something interesting for you all to watch. Just registered a new domain http://www.charles-dickens.org/. It was registered around 5-30pm (GMT) today and had content by 6-00pm. It's 8-00pm and currently there are approx. 600 pages online. The home page of this domain is currently linked to from the 600 pages and 27,000 other pages. Right now the only visitors to the site is me and the Google adwords bot (Mediapartners-Google/2.1). I'll probably add few thousand more pages tonight (if I get the time). What I'm going to be tracking is how long it takes for Google to find the site and for it to show in the index via a allinurl search- allinurl:charles-dickens.org/ That's funny, the home page is already showing for this search, but with no cache. Interesting. So google has managed to add the home page by 7-30pm even though the Google spider hasn't visited! I've got the toolbar installed and google ads (the latter isn't supposed to be a help). I don't submit sites to google, I rely on links for indexing. So either it's been added because I use the toolbar or because Google has spidered one (or more) of the 27,000 pages linking to it and added the home page without spidering it. Either way that was fast. Will try to check the sites allinurl: search every hour and would be interested if anyone else checks when they see more pages listed, especially with a cache since that means a page has been fully spidered and indexed. I've never checked a pages indexing speed in this much detail, best I've seen is a round 12hrs, but could of been much earlier since I wasn't checking on a regular basis. David |
put servoces.
BB
#4
| |||
| |||
|
|
Just registered a new domain http://www.charles-dickens.org/. It was registered around 5-30pm (GMT) today and had content by 6-00pm. It's 8-00pm and currently there are approx. 600 pages online. Slow day then, huh? BB |
That sites basically finished now since I've ran out of Dickens books
to add. 7300 odd pages, not bad for 4 or 5 hours work.
David
--
http://www.seo-serps.com
#5
| |||
| |||
|
#6
| |||
| |||
|
|
That sites basically finished now since I've ran out of Dickens books to add. 7300 odd pages, not bad for 4 or 5 hours work. |
read the odd pages *and* the even pages!
#7
| |||
| |||
|
| www.seo-serps.com <ooar123SP (AT) AMntlworld (DOT) com> says... That sites basically finished now since I've ran out of Dickens books to add. 7300 odd pages, not bad for 4 or 5 hours work. What a useless site! I didn't even look at it, because I want to read the odd pages *and* the even pages! |
David
--
http://www.seo-serps.com
#8
| |||
| |||
|
|
On Sun, 13 Jun 2004 06:21:44 -0700, Guy Macon http://www.guymacon.com> wrote: www.seo-serps.com <ooar123SP (AT) AMntlworld (DOT) com> says... That sites basically finished now since I've ran out of Dickens books to add. 7300 odd pages, not bad for 4 or 5 hours work. What a useless site! I didn't even look at it, because I want to read the odd pages *and* the even pages! What are you going on about? |
And, anyway, Guy ought to be able to write a little routine to simply add 1
to each page number - Bingo! Even pages.
--
PeterMcC
If you feel that any of the above is incorrect,
inappropriate or offensive in any way,
please ignore it and accept my apologies.
#9
| |||||
| |||||
|
|
Hi, Here is something interesting for you all to watch. Just registered a new domain http://www.charles-dickens.org/. It was registered around 5-30pm (GMT) today and had content by 6-00pm. It's 8-00pm and currently there are approx. 600 pages online. The home page of this domain is currently linked to from the 600 pages and 27,000 other pages. Right now the only visitors to the site is me and the Google adwords bot (Mediapartners-Google/2.1). |
Is that not considered a spider for indexing in google ??
Quote:
|
I'll probably add few thousand more pages tonight (if I get the time). |
allow that.
Quote:
|
What I'm going to be tracking is how long it takes for Google to find the site and for it to show in the index via a allinurl search- allinurl:charles-dickens.org/ That's funny, the home page is already showing for this search, but with no cache. Interesting. So google has managed to add the home page by 7-30pm even though the Google spider hasn't visited! I've got the toolbar installed and google ads (the latter isn't supposed to be a help). I don't submit sites to google, I rely on links for indexing. So either it's been added because I use the toolbar or because Google has spidered one (or more) of the 27,000 pages linking to it and added the home page without spidering it. |
Google has just the title in the index so it is beginning to
build a more complete index, but I think the question you're
wondering about is how did google pick that up if googlebot
hasn't spidered unless that AdWords spider or toolbar did it.
If it turns out that it was due to using a browser with the
toolbar that would be interesting to know.
Quote:
|
Either way that was fast. Will try to check the sites allinurl: search every hour and would be interested if anyone else checks when they see more pages listed, especially with a cache since that means a page has been fully spidered and indexed. I've never checked a pages indexing speed in this much detail, best I've seen is a round 12hrs, but could of been much earlier since I wasn't checking on a regular basis. |
Quote:
|
David |
Jon
#10
| |||
| |||
|
|
Guy Macon <http://www.guymacon.com> wrote: www.seo-serps.com <ooar123SP (AT) AMntlworld (DOT) com> says... That sites basically finished now since I've ran out of Dickens books to add. 7300 odd pages, not bad for 4 or 5 hours work. What a useless site! I didn't even look at it, because I want to read the odd pages *and* the even pages! What are you going on about? |
purposely misunderstood (for the sake of the joke) that as
meaning pages 1,3,5,7... and (for the sake of the joke)
complained about you not adding pages 2.4.6.8...
Having to explain a joke means that the joketeller didn't
tell the joke very well.
|
« Previous Thread
|
Next Thread »
| Thread Tools | |
| Display Modes | |
Linear Mode
| |
Powered by vBulletin Version 3.5.4
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.