Re: Not reindexed in Google - yet

On Mon, 6 Sep 2004 17:14:38 +0200, "René Løweneck" <rene (AT) loweneck (DOT) dk>
wrote:

So, my question is : Has anyone in this newsgroup experienced in real life,
to rule a domain out of Google and then get it reindexed ?

Could you clarify what you mean by "rule a domain out"?

David
--
http://www.search-engine-optimization-services.co.uk/

yeah i got banned, fixed the problem emailed google and it took 3 months to
get re-inserted.

mark

On Mon, 6 Sep 2004 18:33:58 +0200, "René Løweneck" <rene (AT) loweneck (DOT) dk>
wrote:

Hi David !

"SEO Dave" <seo-daveSP (AT) AMsearch-engine-optimization-services (DOT) co.uk> wrote in
message news:191pj09pm82ts7mvn9nn72o7485k6je35i (AT) 4ax (DOT) com...
On Mon, 6 Sep 2004 17:14:38 +0200, "René Løweneck" <rene (AT) loweneck (DOT) dk
wrote:

Could you clarify what you mean by "rule a domain out"?

What I meant was, using the robots.txt file in the web root, to exclude (one
specific or all) robots from indexing the entire domain. Like this

--- content of my robots.txt file in the web root 20-03-2004 ---
# go away
User-agent: *
Disallow: /
--- end of content ---

Understood now.

Normally, you would only rule out subdirectories or old filenames, using the
robots.txt file.
Read more about robots.txt here : http://www.robotstxt.org/wc/norobots.html

I however, excluded the robots from indexing the entire domains. I hope that
clarified it for you, David. If not, please ask again.


Kind Regards

Not tried the above, so can't give you a real life example. Would be
interested to know how it goes though.

I would guess the speed will be based on the number and quality of the
links to the domain. If it has no links to it then it would be like
putting up a new site, when I've added new domains they get spidered
and added to the index within a few days since I add a lot of links to
them so bots find them easily.

If the domain is linked to from a page that hardly ever gets spidered
then you are at the mercy of when the bot happens to hit that page.
Could be one day, could be 2 months, no one knows until it happens.

So more links you have to an excluded domain the quicker it should be
indexed when you remove the exclusion.

BTW The domain might already be in the index. Google will add a page
to it's index without a cache if it's got enough links to it.

Would I be right in assuming that based on your robots.txt examples
date above you removed the disallow back in March from a domain and
it's not being indexed yet?

If you post the URL we can check to see if there is a problem.

David
--
http://www.search-engine-optimization-services.co.uk/

On Tue, 7 Sep 2004 08:39:48 +0200, "René Løweneck" <rene (AT) loweneck (DOT) dk>
wrote:

Hi David !

"SEO Dave" <seo-daveSP (AT) AMsearch-engine-optimization-services (DOT) co.uk> wrote in
message news:hc3qj0hmntgk4ujevisf2s2s4l4c6qn36n (AT) 4ax (DOT) com...
On Mon, 6 Sep 2004 18:33:58 +0200, "René Løweneck" <rene (AT) loweneck (DOT) dk
wrote:

Not tried the above, so can't give you a real life example. Would be
interested to know how it goes though.

Ok.

I would guess the speed will be based on the number and quality of the
links to the domain. If it has no links to it then it would be like
putting up a new site, when I've added new domains they get spidered
and added to the index within a few days since I add a lot of links to
them so bots find them easily.

Just what I have thought, when I originally ruled out the domains. I thought
there would have gone one or two months, and then the domains would have
been reindexed as new sites. But hey, I'm just a stupid dane ;-))

About the amount of inbound links, there is not that many. I do not have
that many domains. In fact, I have only three domains. Two of them, being
out of the Google index. The third one, is relatively new and works as a
kind of mirrorsite to my main site (which is not in the Google index). And
doing pretty well btw.

Mirror site? Are we talking duplicated content here?

BB

If the domain is linked to from a page that hardly ever gets spidered
then you are at the mercy of when the bot happens to hit that page.
Could be one day, could be 2 months, no one knows until it happens.
So more links you have to an excluded domain the quicker it should be
indexed when you remove the exclusion.

Nice theory, but on the 20th this month, there have gone six months out of
the Google index :-((

BTW The domain might already be in the index. Google will add a page
to it's index without a cache if it's got enough links to it.

Maybe, but then you would see it in a inurl:domain. My domains simply seems
to be unknown to Google.

But the strangest thing though,- Google visit the domains plenty of times
each day and ocasionally try to get some of the old pages from a half year
ago !
So old links tends to remain in Google for a long time. A very long time
indeed ???????????

Would I be right in assuming that based on your robots.txt examples
date above you removed the disallow back in March from a domain and
it's not being indexed yet?

Yes. The exclusion in the robots.txt was in effect one week. After that, I
only excluded some subdirectories.

If you post the URL we can check to see if there is a problem.

Ok David, here is the urls :

http://www.loweneck.dk
http://www.lowesite.dk


Kind Regards

On Tue, 7 Sep 2004 10:27:11 +0200, "René Løweneck" <rene (AT) loweneck (DOT) dk>
wrote:

Hi Bill !

"Big Bill" <kruse (AT) cityscape (DOT) co.uk> wrote in message
news:q0sqj0l5peb0qai28vbtlksfsn220fvj06 (AT) 4ax (DOT) com...
On Tue, 7 Sep 2004 08:39:48 +0200, "René Løweneck" <rene (AT) loweneck (DOT) dk
wrote:

Mirror site? Are we talking duplicated content here?

Yes, but not all pages are on the mirrorsite though. I know you are about to
say, but that does not mean a thing, because the new domain has just been
running as a partly mirror for the last month or so.


Kind Regards

I think it may well mean something....

BB

On Tue, 7 Sep 2004 17:57:10 +0200, "René Løweneck" <rene (AT) loweneck (DOT) dk>
wrote:

Hi David !

"SEO Dave" <seo-daveSP (AT) AMsearch-engine-optimization-services (DOT) co.uk> wrote in
message news:8garj0lhd72b0mop0lsfnpp42nb6p4101n (AT) 4ax (DOT) com...
On Tue, 7 Sep 2004 08:39:48 +0200, "René Løweneck" <rene (AT) loweneck (DOT) dk
wrote:

This http://www.top-senior.dk/ PR3 looks like your best link.

A funny thing about this, is that http://www.top-senior.dk/default.asp has a
PR4. It's the same page and the server uses default.asp as default. Google
makes two seperate pages out of the frontpage (with seperate PR's). I have
seen this on other domains (not mine nor hosted by me) as well. It could
seem, that Google would only recognice . htm or .html as an alias for the
domainname itself.

Sometimes Google doesn't combine them. Difficult to know why one is
and one isn't combined.

Both domains have inbound links from DMOZ. I don't know the PR of these
links though.

I would say it's a lack of quality links. You can find pages with
better links than your sites have that aren't in Google.

Maybe you are right, but where do I find such quality links ?

One good link could get you in.

But where do I find that link ? ;-))

Directories related to your site. Friends sites, trade something for a
link. I didn't really look at your sites content, just the code but
noticed you where doing something about a CMS. Give it away for a link
on each page it's used on.

You could always buy a small number of links to get you started. Avoid
sites selling site wide links though, lots of these aren't
transferring PR.

Do you have the Google Toolbar? If you visit your site with it on the
browser Google will find your site. There have been reports of sites
getting indexed from just the toolbar!

Yes, I've got the Google Toolbar and http://www.loweneck.dk is my startpage.

site:domain.com tends to give consistent results these days. Looks
like they have removed the allinurl: function from Google.

But the inurl:domain.com is still around.

So it is. I'd not used that one before, always used allinurl: until it
stopped working recently.

Both sites aren't indexed.

Nope ;-))

Generally you would expect to see those pages in the index within a
week. Google doesn't add everything it spiders, there seems to be a
quality cut off point where if a page doesn't have X it's not added. X
being the right number/quality of links.

You could be right, but then again, I don't know where to find any quality
links.

The only way you'll know if this is the problem is to add some decent
PR links to the site (ideally PR5). If that doesn't get the site
indexed there is a problem somewhere else.

Come to think about it, the ISP has been extremely unstable this summer. I
have therefore just got myself a new ISP. So that just might do the trick
;-))

Might be a factor.

You can easily check your code is OK by signing up a free domain
somewhere and uploading it. Add a few reasonable PR spammy links (easy
to find) and see if it's indexed. If it is indexed you might want to
look at the server that there isn't something configured incorrectly.

I think my code is ok. Top-senior.dk and my outside developmentsite
(mirrorsite) is doing pretty well.

There are a few small errors like using ' instead of " on some of your
links, but nothing major.

To avoid mirror problems delete the free site after use or at least
add different content.

The page content is sligthly different.

Cut it down to as what I've removed is either ignored or pointless

You are right, that Google tends to ignore most of the Meta tags. But there
are other searchengines too. So I prefer to lieve the Meta tags untouched,
for now at least ;-))

Most of what I removed is ignored by all major engines. The DC ones
are obsolete.

Also title="bunch of words" is ignored by Google.

Titles on images you mean ? I did'nt know that. Thanks :-))

title="keywords" on anything is ignored by google. So is the comments
area.

PS.: Do you handcode the pages on your own domain ?

Yes, though I tend to create templates by hand and use other tools to
make multiple pages. My favourite HTML editor is a freeware Notepad
replacement program called NoteSpad. Also like Dreamweaver for it's
ability to update an entire site in one go, but wouldn't touch the
design bit (WYSIWYG bit) with a 100ft barge pole, so use it as an
expensive text editor :-))

David
--
http://www.search-engine-optimization-services.co.uk/