Server-side includes and search engines

I've read that search engines don't spider links to
dynamically-generated pages, like this:

http://www.mysite.com/products.asp?id=1

Are they able to spider links that are on a page via a server-side
include? For example, if I want to put the code for all my navigation
into a file called

mainnav.asp

and then pull that into the file using

<!--# include file="mainnav.asp" -->

The links in the mainnav.asp file are plain old text links.

Thanks in advance.

Win
-----------
Win Day, Multimedia Developer
Words Plus
http://www.wordsplus.net
winday (AT) REMOVETHISwordsplus (DOT) net

Win Day <winday (AT) removethiswordsplus (DOT) net> wrote:

I've read that search engines don't spider links to
dynamically-generated pages, like this:

http://www.mysite.com/products.asp?id=1

My experience with Google is that it will crawl and index such pages
in this form, but the more args you have to the script, the less
likely Google is to index further. So this page

http://www.mysite.com/products.asp?id=1&name=John

is not liked by Google as much as the first example.

Are they able to spider links that are on a page via a server-side
include?

Robots cannot tell the difference between a page that is pure HTML and
one that is generated by a server-side include. All they see is the
HTML code that is sent back to them.

Another approach to showing clean URL's besides a server-side include
is to use mod_name if you are using Apache to rename the URL, but it
looks like you are using MS IIS...

--
I.

You've been reading old articles Dynamic pages are okay as long as
you don't use session ids.

And yes, they'll pick up the SSI fine.
------------------------------
Posted via http://www.searchguild.com/

Win Day wrote:

Are they able to spider links that are on a page via a server-side
include?

Yes. spiders only receive the post processed HTML, so can't tell whether you
are using SSI or not.


--
Iso.
FAQs: http://html-faq.com http://alt-html.org http://allmyfaqs.com/
Recommended Hosting: http://www.affordablehost.com/
Web Standards: http://www.webstandards.org/

Gazing into my crystal ball I observed "NorwichLad" <usenet (AT) periods (DOT) info>
writing in news:1062801286.79382.0 (AT) eunomia (DOT) uk.clara.net:

"Win Day" <winday (AT) REMOVETHISwordsplus (DOT) net> wrote in message
news:28rhlvsn32himejrlp9uk3i4fse851ha0h (AT) 4ax (DOT) com...
I've read that search engines don't spider links to
dynamically-generated pages, like this:

http://www.mysite.com/products.asp?id=1

Are they able to spider links that are on a page via a server-side
include? For example, if I want to put the code for all my navigation
into a file called

I used to have this problem for some of my sites, in particular, one I
run for a UK Veterinary magazine.

I solved the problem using Apache and mod_rewrite

Although the real pages are
www.site.com/news.php?id=100

The links in the site are to:
www.site.com/news/100.html

These are also accessable from the outside world, too. mod_rewrite
automatically converts www.site.com/news/number.html to
www.site.com/news.php?id=number

Look up mod_rewrite in google, it really is a life saver!

Mod_rewrite will not work on IIS servers, and the OP seems to be using IIS.


--
Adrienne Boswell
Please respond to the group so others can share
http://www.arbpen.com

On Sat, 06 Sep 2003 01:44:12 GMT, Adrienne <arbpen (AT) sbcglobal (DOT) net>
wrote:

Gazing into my crystal ball I observed "NorwichLad" <usenet (AT) periods (DOT) info
writing in news:1062801286.79382.0 (AT) eunomia (DOT) uk.clara.net:


"Win Day" <winday (AT) REMOVETHISwordsplus (DOT) net> wrote in message
news:28rhlvsn32himejrlp9uk3i4fse851ha0h (AT) 4ax (DOT) com...
I've read that search engines don't spider links to
dynamically-generated pages, like this:

http://www.mysite.com/products.asp?id=1

Are they able to spider links that are on a page via a server-side
include? For example, if I want to put the code for all my navigation
into a file called

I used to have this problem for some of my sites, in particular, one I
run for a UK Veterinary magazine.

I solved the problem using Apache and mod_rewrite

Although the real pages are
www.site.com/news.php?id=100

The links in the site are to:
www.site.com/news/100.html

These are also accessable from the outside world, too. mod_rewrite
automatically converts www.site.com/news/number.html to
www.site.com/news.php?id=number

Look up mod_rewrite in google, it really is a life saver!




Mod_rewrite will not work on IIS servers, and the OP seems to be using IIS.

Very true. Anybody know if there's sometihng similar for IIS?

Win
-------------
Win Day, Multimedia Developer
Words Plus
http://www.wordsplus.net
winday (AT) REMOVETHISwordsplus (DOT) net

Gazing into my crystal ball I observed Win Day
<winday (AT) REMOVETHISwordsplus (DOT) net> writing in
news:chnilvk2uuhp5j4n6m7it5er7lf3k9adji (AT) 4ax (DOT) com:

On Sat, 06 Sep 2003 01:44:12 GMT, Adrienne <arbpen (AT) sbcglobal (DOT) net
wrote:

Gazing into my crystal ball I observed "NorwichLad"
usenet (AT) periods (DOT) info> writing in
news:1062801286.79382.0 (AT) eunomia (DOT) uk.clara.net:


"Win Day" <winday (AT) REMOVETHISwordsplus (DOT) net> wrote in message
news:28rhlvsn32himejrlp9uk3i4fse851ha0h (AT) 4ax (DOT) com...
I've read that search engines don't spider links to
dynamically-generated pages, like this:

http://www.mysite.com/products.asp?id=1

Are they able to spider links that are on a page via a server-side
include? For example, if I want to put the code for all my
navigation into a file called

I used to have this problem for some of my sites, in particular, one
I run for a UK Veterinary magazine.

I solved the problem using Apache and mod_rewrite

Although the real pages are www.site.com/news.php?id=100

The links in the site are to: www.site.com/news/100.html

These are also accessable from the outside world, too. mod_rewrite
automatically converts www.site.com/news/number.html to
www.site.com/news.php?id=number

Look up mod_rewrite in google, it really is a life saver!




Mod_rewrite will not work on IIS servers, and the OP seems to be using
IIS.

Very true. Anybody know if there's sometihng similar for IIS?

I haven't used it, but there's IIS Rewrite
http://www.qwerksoft.com/products/iisrewrite/ .




--
Adrienne Boswell
Please respond to the group so others can share
http://www.arbpen.com

If you are feeling brave you can move onto .NET and this includes a class
for rewriting URL's. There are a number of articles floating about
explaining how to use it.

--
Martyn Fewtrell
mfewtrell (AT) networkclub (DOT) co.uk

http://www.networkclub.co.uk


"Adrienne" <arbpen (AT) sbcglobal (DOT) net> wrote

Gazing into my crystal ball I observed Win Day
winday (AT) REMOVETHISwordsplus (DOT) net> writing in
news:chnilvk2uuhp5j4n6m7it5er7lf3k9adji (AT) 4ax (DOT) com:

On Sat, 06 Sep 2003 01:44:12 GMT, Adrienne <arbpen (AT) sbcglobal (DOT) net
wrote:

Gazing into my crystal ball I observed "NorwichLad"
usenet (AT) periods (DOT) info> writing in
news:1062801286.79382.0 (AT) eunomia (DOT) uk.clara.net:


"Win Day" <winday (AT) REMOVETHISwordsplus (DOT) net> wrote in message
news:28rhlvsn32himejrlp9uk3i4fse851ha0h (AT) 4ax (DOT) com...
I've read that search engines don't spider links to
dynamically-generated pages, like this:

http://www.mysite.com/products.asp?id=1

Are they able to spider links that are on a page via a server-side
include? For example, if I want to put the code for all my
navigation into a file called

I used to have this problem for some of my sites, in particular, one
I run for a UK Veterinary magazine.

I solved the problem using Apache and mod_rewrite

Although the real pages are www.site.com/news.php?id=100

The links in the site are to: www.site.com/news/100.html

These are also accessable from the outside world, too. mod_rewrite
automatically converts www.site.com/news/number.html to
www.site.com/news.php?id=number

Look up mod_rewrite in google, it really is a life saver!




Mod_rewrite will not work on IIS servers, and the OP seems to be using
IIS.

Very true. Anybody know if there's sometihng similar for IIS?


I haven't used it, but there's IIS Rewrite
http://www.qwerksoft.com/products/iisrewrite/ .




--
Adrienne Boswell
Please respond to the group so others can share
http://www.arbpen.com

On Sat, 06 Sep 2003 08:41:25 GMT, Adrienne <arbpen (AT) sbcglobal (DOT) net>
wrote:

Gazing into my crystal ball I observed Win Day
winday (AT) REMOVETHISwordsplus (DOT) net> writing in
news:chnilvk2uuhp5j4n6m7it5er7lf3k9adji (AT) 4ax (DOT) com:

On Sat, 06 Sep 2003 01:44:12 GMT, Adrienne <arbpen (AT) sbcglobal (DOT) net
wrote:

Gazing into my crystal ball I observed "NorwichLad"
usenet (AT) periods (DOT) info> writing in
news:1062801286.79382.0 (AT) eunomia (DOT) uk.clara.net:


"Win Day" <winday (AT) REMOVETHISwordsplus (DOT) net> wrote in message
news:28rhlvsn32himejrlp9uk3i4fse851ha0h (AT) 4ax (DOT) com...
I've read that search engines don't spider links to
dynamically-generated pages, like this:

http://www.mysite.com/products.asp?id=1

Are they able to spider links that are on a page via a server-side
include? For example, if I want to put the code for all my
navigation into a file called

I used to have this problem for some of my sites, in particular, one
I run for a UK Veterinary magazine.

I solved the problem using Apache and mod_rewrite

Although the real pages are www.site.com/news.php?id=100

The links in the site are to: www.site.com/news/100.html

These are also accessable from the outside world, too. mod_rewrite
automatically converts www.site.com/news/number.html to
www.site.com/news.php?id=number

Look up mod_rewrite in google, it really is a life saver!




Mod_rewrite will not work on IIS servers, and the OP seems to be using
IIS.

Very true. Anybody know if there's sometihng similar for IIS?


I haven't used it, but there's IIS Rewrite
http://www.qwerksoft.com/products/iisrewrite/ .

Thanks, Adrienne , I'll have a look.

Win
------------
Win Day, Multimedia Developer
Words Plus
http://www.wordsplus.net
winday (AT) REMOVETHISwordsplus (DOT) net

On Sat, 6 Sep 2003 10:42:12 +0000 (UTC), "Martyn Fewtrell"
<mfewtrell (AT) networkclub (DOT) co.uk> wrote:

If you are feeling brave you can move onto .NET and this includes a class
for rewriting URL's. There are a number of articles floating about
explaining how to use it.

I don't think I'm that brave yet! I did buy some books, but haven't
had the chance to crack them yet.

!in
-----------
Win Day, Multimedia Developer
Words Plus
http://www.wordsplus.net
winday (AT) REMOVETHISwordsplus (DOT) net