HighDots Forums  

Spiders out of control

Search Engine Optimization Discussion about SEO/Search Engine Optimization (alt.internet.search-engines)


Discuss Spiders out of control in the Search Engine Optimization forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
DesignGuy
 
Posts: n/a

Default Spiders out of control - 09-04-2003 , 11:26 PM






This afternoon and evening one of my sites was spidered (heavily; several
pages per second) by two entities:

64.68.115.166 (unassigned.webex.com)
OrgName: WebEx Inc
OrgID: WEBEXI
Address: 110 Rose Orchard Way
City: San Jose
StateProv: CA
PostalCode: 94401
Country: US


61.149.34.108
OrgName: Asia Pacific Network Information Centre
OrgID: APNIC
Address: PO Box 2131
City: Milton
StateProv: QLD
PostalCode: 4064
Country: AU

Since search engine usually wait at least a few seconds between page
requests, I'm guessing neither one of these are a legitimate search engine.
But can anyone shed any light on what/who these spiders belong to?

Thanks in advance....




Reply With Quote
  #2  
Old   
T T
 
Posts: n/a

Default Re: Spiders out of control - 09-05-2003 , 05:52 AM






I've never heard of WebEx but the Asia Pacific Network is most likely a
spambot looking for e-mail addresses to send you spam! I had such a big
problem with them that I completely removed my e-mail address from the site
and now have a php reply form instead!


"DesignGuy" <dontbother (AT) nowhere (DOT) com> wrote

Quote:
This afternoon and evening one of my sites was spidered (heavily; several
pages per second) by two entities:

64.68.115.166 (unassigned.webex.com)
OrgName: WebEx Inc
OrgID: WEBEXI
Address: 110 Rose Orchard Way
City: San Jose
StateProv: CA
PostalCode: 94401
Country: US


61.149.34.108
OrgName: Asia Pacific Network Information Centre
OrgID: APNIC
Address: PO Box 2131
City: Milton
StateProv: QLD
PostalCode: 4064
Country: AU

Since search engine usually wait at least a few seconds between page
requests, I'm guessing neither one of these are a legitimate search
engine.
But can anyone shed any light on what/who these spiders belong to?

Thanks in advance....






Reply With Quote
  #3  
Old   
Foxglove54321
 
Posts: n/a

Default Re: Spiders out of control - 09-05-2003 , 09:22 AM



Quote:
I've never heard of WebEx but the Asia Pacific Network is most likely a
spambot looking for e-mail addresses to send you spam!
You can test this theory by setting up a fake directory excluded by robots.txt,
and see if it goes in there. There are spider traps available that are intended
to choke the email harvesters, but I don't think I'd want to bother with one
because it smacks of throwing good bandwidth after bad.

--
Alice Woolley
http://www.insidethebubble.co.uk/
Inside the Bubble - autism information


Reply With Quote
  #4  
Old   
James Taylor
 
Posts: n/a

Default Re: Spiders out of control - 09-05-2003 , 10:55 AM



"T T" <tool (AT) toolbox (DOT) net> wrote

Quote:
I've never heard of WebEx but the Asia Pacific Network is most likely a
spambot looking for e-mail addresses to send you spam! I had such a big
problem with them that I completely removed my e-mail address from the
site
and now have a php reply form instead!


"DesignGuy" <dontbother (AT) nowhere (DOT) com> wrote in message
news:vnT5b.269397$It4.125917 (AT) rwcrnsc51 (DOT) ops.asp.att.net...
This afternoon and evening one of my sites was spidered (heavily;
several
pages per second) by two entities:

64.68.115.166 (unassigned.webex.com)
OrgName: WebEx Inc
OrgID: WEBEXI
Address: 110 Rose Orchard Way
City: San Jose
StateProv: CA
PostalCode: 94401
Country: US
The only WebEx I have heard of are the video conferencing people at;
http://www.webex.com/services_confer...-overview.html

James Taylor
www.AICompany.com

Quote:




Reply With Quote
  #5  
Old   
DesignGuy
 
Posts: n/a

Default Re: Spiders out of control - 09-05-2003 , 10:58 AM



Thanks for your help and suggestions... I'll monitor it and if it occurs
again I'll disallow their IP address blocks. I agree that wasting bandwidth
for "revenge" purposes is a bad thing, enticing as though it might be.

Teoma and Alexa have also been spidering like crazy - but at least they wait
a few seconds between pages, and I'll (hopefully) get some benefit down the
road.





Reply With Quote
  #6  
Old   
Brothermark
 
Posts: n/a

Default Re: Spiders out of control - 09-06-2003 , 10:09 PM



Quote:
I had such a big
problem with them that I completely removed my e-mail address from the
site
and now have a php reply form instead!
But people regularly input their email address incorrectly.
It seems best to just fight the spambots so that you can still list your
email on the website

mark




Reply With Quote
  #7  
Old   
info
 
Posts: n/a

Default Re: Spiders out of control - 09-08-2003 , 04:48 AM



that's why you require them to input the e-mail address 2x and have it check
to be sure they are the same.
a form is how you can fight the spam bots because otherwise you will lose
big time.

if you put a mailto link on your site you will get spam.

one solution is to use unicode to encode your mailto link just go here for
how to do it and don't worry this site is not a spam site

http://fantomaster.com/fantomasSuite...shieldsv-e.cgi


"Brothermark" <yonnermark (AT) hotmail (DOT) com> wrote

Quote:
I had such a big
problem with them that I completely removed my e-mail address from the
site
and now have a php reply form instead!

But people regularly input their email address incorrectly.
It seems best to just fight the spambots so that you can still list your
email on the website

mark





Reply With Quote
  #8  
Old   
pHamous
 
Posts: n/a

Default Re: Spiders out of control - 09-10-2003 , 10:54 AM



"info" <info (AT) anonymous (DOT) com> wrote


Hi info..

Quote:
if you put a mailto link on your site you will get spam.

one solution is to use unicode to encode your mailto
Sure. What about script in the body og the HTML? e.g

<script language="JavaScript" type="text/javascript">
<!--
var user = 'info' ;
var domain = 'anonymous.com' ;
document.write('<a href="mailto:'+user+'@'+domain+'">'+user+'@'+domai n+'</a>');
//-->
</script>


Try it and see?

pHamous


Reply With Quote
  #9  
Old   
Adrienne
 
Posts: n/a

Default Re: Spiders out of control - 09-10-2003 , 12:10 PM



Gazing into my crystal ball I observed google (AT) phamous (DOT) net (pHamous)
writing in news:de244f51.0309100654.46fd7450 (AT) posting (DOT) google.com:

Quote:
"info" <info (AT) anonymous (DOT) com> wrote in message
news:<WnX6b.26606$Dg7.19010 (AT) nwrddc02 (DOT) gnilink.net>...

Hi info..

if you put a mailto link on your site you will get spam.

one solution is to use unicode to encode your mailto

Sure. What about script in the body og the HTML? e.g

script language="JavaScript" type="text/javascript"
!--
var user = 'info' ;
var domain = 'anonymous.com' ;
document.write('<a
href="mailto:'+user+'@'+domain+'">'+user+'@'+domai n+
'</a>');
//--
/script


Try it and see?

pHamous

Doesn't work for users with Javascript disabled, eg. real people.

--
Adrienne Boswell
Please respond to the group so others can share
http://www.arbpen.com


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.4
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.