Browser Check Redirection is Catching Search Engines

My website currently checks for browsers. Things don't work properly
if the user doesn't have a browser IE5 or higher or Nav6 or higher. I
forward them to a browser upgrade page, if there browser doesn't meet
specs because I don't want to deal with bugs for older browser
versions.

The problem is that search engines are being caught in the same code
(because the user-agent isn't mozilla 6 or higher and is not msie 5 or
higher).

First thing would be to admit that Google has a very high percentage
of the search engine market .. so if I can fix it for google, I am way
further ahead, compared to today. I am planning to check for
"googlebot" as a substring of user-agent (back-end code) and bypass
the whole browser-check code in this case.

Presumably this will solve the problem for Google?

My question is: what set of user-agent strings/substrings should I
watch out for so the browser check can be skipped for the other search
engines? I've searched newsgroups and on a few occasions heard 1-2
other mentioned.

The realize that the list will be incomplete and get out-of-date ..
but at the moment none of the search engines are indexing my site, so
even a static/stale list will be a big improvement.

On 29 Dec 2003 11:33:12 -0800, James <jw697569 (AT) hotmail (DOT) com> wrote:

My website currently checks for browsers. Things don't work properly
if the user doesn't have a browser IE5 or higher or Nav6 or higher. I
forward them to a browser upgrade page, if there browser doesn't meet
specs because I don't want to deal with bugs for older browser
versions.

When I encounter a site that attempts to redirect me to another URL without
asking me first, my browser automatically breaks the http connection and
enters the original URL in a blocklist.

Hate to be the one to break it to you, but we don't have to change our
browsers just because you want to add another layer of irrelevant eye-candy
to your site.

And you do NOT have the right to redirect a person's browser without asking
their permission.


On the other hand, it is easy enough to TELL a server you are running IE5
and to obtain whatever useful information is there without seeing the site
as the webmaster wishes it to be seen.


The arrogance of some webmasters is just fucking unbelievable.


AC

James <jw697569 (AT) hotmail (DOT) com> wrote:

My website currently checks for browsers. Things don't work properly
if the user doesn't have a browser IE5 or higher or Nav6 or higher. I
forward them to a browser upgrade page, if there browser doesn't meet
specs because I don't want to deal with bugs for older browser
versions.

The problem is that search engines are being caught in the same code
(because the user-agent isn't mozilla 6 or higher and is not msie 5 or
higher).

First thing would be to admit that Google has a very high percentage
of the search engine market .. so if I can fix it for google, I am way
further ahead, compared to today. I am planning to check for
"googlebot" as a substring of user-agent (back-end code) and bypass
the whole browser-check code in this case.

Presumably this will solve the problem for Google?

My question is: what set of user-agent strings/substrings should I
watch out for so the browser check can be skipped for the other search
engines? I've searched newsgroups and on a few occasions heard 1-2
other mentioned.

The realize that the list will be incomplete and get out-of-date ..
but at the moment none of the search engines are indexing my site, so
even a static/stale list will be a big improvement.

Do it backwards: instead of looking for only the latest browsers, look
for old browsers that you do not want to support. Specifically look
for MSIE and Netscape versions that are too old to work for your site,
then redirect these users to the "upgrade" page.

If you do it this way, any new browser version or robot will be able
to crawl your site. You will not have to keep updating your site once
MSIE 7.0 is released or whatever. But no more "old browsers" will be
coming out, presumably. It shouldn't take you much time to work up a
list of "old browser" signatures that you can key off of - probably
someone out there has already done it, if you search the web a bit.

--
I.

"Alan Connor" <zzzzzz (AT) xxx (DOT) yyy> wrote

When I encounter a site that attempts to redirect me to another URL
without asking me first, my browser automatically breaks the http
connection and enters the original URL in a blocklist.

Seriously? Since using http redirects is nothing unusual or nafarious, you
must have a huge blocklist. What you're saying is that if a site moves to a
new URL and simply wants to redirect you to the current location, your
browser bans the site. How weird. And what happens when you click a text
ad or any kind of link that has click logging? When the script redirects
you to the site you are looking for after the click is logged, why would you
want your browser to blacklist the link? That is just odd.

--
- Brandon Staggs
http://www.brandonstaggs.com

James wrote:

My website currently checks for browsers. Things don't work properly
...

No they don't. Stop doing it.

--
William Tasso - http://WilliamTasso.com

Brandon wrote:

"Alan Connor" <zzzzzz (AT) xxx (DOT) yyy> wrote in message
news:5K0Ib.10867$lo3.7776 (AT) newsread2 (DOT) news.pas.earthlink.net

When I encounter a site that attempts to redirect me to another URL
without asking me first, my browser automatically breaks the http
connection and enters the original URL in a blocklist.

Seriously? Since using http redirects is nothing unusual or
nafarious, you must have a huge blocklist.

Why the problem? I block all attempts at meta refresh/redirect.

What you're saying is
that if a site moves to a new URL and simply wants to redirect you to
the current location, your browser bans the site.

A 'moved permanently' response is not the same.

--
William Tasso - http://WilliamTasso.com

"William Tasso" <news27 (AT) tbdata (DOT) com> wrote

Brandon wrote:
"Alan Connor" <zzzzzz (AT) xxx (DOT) yyy> wrote in message
news:5K0Ib.10867$lo3.7776 (AT) newsread2 (DOT) news.pas.earthlink.net

When I encounter a site that attempts to redirect me to another URL
without asking me first, my browser automatically breaks the http
connection and enters the original URL in a blocklist.

Seriously? Since using http redirects is nothing unusual or
nafarious, you must have a huge blocklist.

Why the problem? I block all attempts at meta refresh/redirect.

What you're saying is
that if a site moves to a new URL and simply wants to redirect you to
the current location, your browser bans the site.

A 'moved permanently' response is not the same.

As I quoted above from the OP, he said he blocks *any* site that redirects
him without "asking" him first. This presumably includes 30x redirects.

Even if you only block meta refresh redirects, I still don't "get" why. Who
cares? If you don't like the site you are redirected to, then close the web
page. There are many servers that don't allow people to put up .htaccess
files for redirects and a meta refresh is the way to handle it. Not to
mention sites that use refresh for obvious reasons, such as some news
headline sites.


--
- Brandon Staggs
http://www.brandonstaggs.com

Brandon wrote:

Even if you only block meta refresh redirects, I still don't "get"
why.

The on-topic response is: because refresh/redirect is one of many seo
techniques designed to lure SE users to a page that would not otherwise
qualify for a decent SERP. I have no interest in doing business with
companies using such underhand techniques.

--
William Tasso - http://WilliamTasso.com

In article <bssud0$14s7p$1 (AT) ID-139074 (DOT) news.uni-berlin.de>, William Tasso
<news27 (AT) tbdata (DOT) com> writes

Brandon wrote:

Even if you only block meta refresh redirects, I still don't "get"
why.

The on-topic response is: because refresh/redirect is one of many seo
techniques designed to lure SE users to a page that would not otherwise
qualify for a decent SERP. I have no interest in doing business with
companies using such underhand techniques.

How is this different from a status code 301 or 302 redirection which

includes a hypertext body?

--
Philip Baker

Philip Baker wrote:

In article <bssud0$14s7p$1 (AT) ID-139074 (DOT) news.uni-berlin.de>, William
Tasso <news27 (AT) tbdata (DOT) com> writes
Brandon wrote:

Even if you only block meta refresh redirects, I still don't "get"
why.

The on-topic response is: because refresh/redirect is one of many seo
techniques designed to lure SE users to a page that would not
otherwise qualify for a decent SERP. I have no interest in doing
business with companies using such underhand techniques.

How is this different from a status code 301 or 302 redirection which
includes a hypertext body?

Does google (or any other) index the content of a 301 ? I doubt it, but
will be happy to see proof either way.

--
William Tasso - http://WilliamTasso.com