Google keeps looking for 404 pages

Googlebot requests repeatedly, over a period of months, some nonexistent
pages from a site I manage for which 404 responses are returned.

I used the link: feature to find out what pages link to the missing
pages on my site, and no hits are returned.

So if Google keeps getting 404s, and it isn't finding links elsewhere
that give it reason to think those pages should exist, then why does it
continue to look for these pages?

On Wed, 03 May 2006 15:58:53 -0400, Harlan Messinger
<hmessinger.removethis (AT) comcast (DOT) net> wrote:

Googlebot requests repeatedly, over a period of months, some nonexistent
pages from a site I manage for which 404 responses are returned.

I used the link: feature to find out what pages link to the missing
pages on my site, and no hits are returned.

Try in Yahoo, it's more accurate about backlinks.

So if Google keeps getting 404s, and it isn't finding links elsewhere
that give it reason to think those pages should exist, then why does it
continue to look for these pages?

It is finding links, it just isn't disclosing to you where it finds
those links. I'm guessing.

BB
--

http://www.kruse.co.uk/sandbox.htm
http://www.here-be-posters.co.uk/lithographs.htm
http://www.crystal-liaison.com/willow-tree/index.html

Googlebot requests repeatedly, over a period of months, some nonexistent pages from a site I manage for which 404 responses are returned.

I had a dose of this back in January/February - Google trying to
download a fair number of pages that haven't existed for a couple of
years. They did it several times and the 404s were polluting my
statistics, so I stuck redirects up and they stopped doing it.

I've no idea where the URIs came from that Google was using. No search
on any engine revealed any of them out on the web - they must have come
from some recycled internal Google resource.

Harlan Messinger wrote:

Googlebot requests repeatedly, over a period of months, some
nonexistent pages from a site I manage for which 404 responses are returned.

It won't hurt to verify that the pages actually return 404 status and not
200 due to some custom error page. It is a very common issue but one
that's very easy to fix.
Feed that invalid URL that is supposed to return 404 HTTP header into one
of HTTP header viewers (here's my personal favorite:
http://www.rexswain.com/httpview.html ) See what your server actually
sends. In an ideal case it should be HTTP/1.1•404•Not•Found(CR)(LF) and
then some troubleshooting text if you like.
If it returns 200 or 302 codes, make sure you get rid of the custom error
handling in your CPanel (of you use it). 301 code may be all right, but it
depends on where do you actually redirect it. I personally would not
settle for anything other than 404. if it’s an error, it has to be purged
out of Google index. After all, all the troubleshooting info you can get
later from your error logs.

--
Cheers,
Dmitri
See Site Sig Below


--
##-----------------------------------------------##
Article posted with Web Developer's USENET Archive
http://www.1-script.com/forums/
Web and RSS gateway to your favorite newsgroup -
alt.internet.search-engines - 30434 messages and counting!
##-----------------------------------------------##

Harlan Messinger <hmessinger.removethis (AT) comcast (DOT) net> wrote:

Googlebot requests repeatedly, over a period of months, some nonexistent
pages from a site I manage for which 404 responses are returned.

I used the link: feature to find out what pages link to the missing
pages on my site, and no hits are returned.

Because link: doesn't return all pages. Also, the link can be on a forum
etc. Check referers.

So if Google keeps getting 404s, and it isn't finding links elsewhere
that give it reason to think those pages should exist, then why does it
continue to look for these pages?

See above. I would redirect to the right page.

--
John

Net::Google and Perl: http://johnbokma.com/perl/net-google.html

On Wed, 03 May 2006 21:58:53 +0200, Harlan Messinger
<hmessinger.removethis (AT) comcast (DOT) net> wrote:

Googlebot requests repeatedly, over a period of months, some nonexistent
pages from a site I manage for which 404 responses are returned.

I used the link: feature to find out what pages link to the missing
pages on my site, and no hits are returned.

So if Google keeps getting 404s, and it isn't finding links elsewhere
that give it reason to think those pages should exist, then why does it
continue to look for these pages?

Google doesn't show all pages that links to your page, so lack of links
displayed doesn't mean anything. Look for links using Yahoo.

Just in case: are you sure you don't have these links somewhere on your
site or in the sitemap?

Best,
Borek
--
http://www.chembuddy.com
http://www.ph-meter.info/pH
http://www.terapia-kregoslupa.waw.pl

Borek wrote:

On Wed, 03 May 2006 21:58:53 +0200, Harlan Messinger
hmessinger.removethis (AT) comcast (DOT) net> wrote:

Googlebot requests repeatedly, over a period of months, some
nonexistent pages from a site I manage for which 404 responses are
returned.

I used the link: feature to find out what pages link to the missing
pages on my site, and no hits are returned.

So if Google keeps getting 404s, and it isn't finding links elsewhere
that give it reason to think those pages should exist, then why does
it continue to look for these pages?

Google doesn't show all pages that links to your page, so lack of links
displayed doesn't mean anything. Look for links using Yahoo.

Just in case: are you sure you don't have these links somewhere on your
site or in the sitemap?

Not in the site map, but using Yahoo I did find a couple on some very
old pages, a couple of which have long since been superseded, so some
redirects are in order.

__/ [ Harlan Messinger ] on Wednesday 03 May 2006 23:11 \__

Borek wrote:
On Wed, 03 May 2006 21:58:53 +0200, Harlan Messinger
hmessinger.removethis (AT) comcast (DOT) net> wrote:

Googlebot requests repeatedly, over a period of months, some
nonexistent pages from a site I manage for which 404 responses are
returned.

I used the link: feature to find out what pages link to the missing
pages on my site, and no hits are returned.

So if Google keeps getting 404s, and it isn't finding links elsewhere
that give it reason to think those pages should exist, then why does
it continue to look for these pages?

Google doesn't show all pages that links to your page, so lack of links
displayed doesn't mean anything. Look for links using Yahoo.

Just in case: are you sure you don't have these links somewhere on your
site or in the sitemap?

Not in the site map, but using Yahoo I did find a couple on some very
old pages, a couple of which have long since been superseded, so some
redirects are in order.

Just to set a certain record straight *smile*, Yahoo's link count may be
higher, but the user is not able to view them all. They also used to (unless
they still do) return a high(er) estimate of hits for merely any query, but
as you explore the SERP's, you are obstructed at a certain stage. If it is
out of sight, it may prove less useful, though it remains a better gauge for
one's links, I agree. "link:URL" queries is (at least) one territory where
Google are not at the forefront.

It's usually better to mix n' match. See:

http://www.pandia.com/sew/208-britis...practices.html

,----[ Snippets ]

British study of search practices

As many as 20 percent of respondents use four or more search
engines, a British study reports.

According to a study from the British marketing company Harvest
Digital only 24 percent of British searchers use only one
search engine.
`----

Best wishes,

Roy

--
Roy S. Schestowitz | "Stand for nothing and you will fall for anything"
http://Schestowitz.com | SuSE Linux ¦ PGP-Key: 0x74572E8E
6:05am up 6 days 13:02, 13 users, load average: 0.44, 0.68, 0.62
http://iuron.com - Open Source knowledge engine project