HighDots Forums  

Is this robots.txt file doing what I think it is doing?

Website Design comp.infosystems.www.authoring.site-design


Discuss Is this robots.txt file doing what I think it is doing? in the Website Design forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
 
Posts: n/a

Default Is this robots.txt file doing what I think it is doing? - 03-18-2005 , 08:45 PM






Someone please validate that this 'robots.txt' file that I put in the root,
allows all robots to look at my entire site, but not the 3 folders and it's
subfolders. Thank you.


# /robots.txt file for http://www.OfficeActivate.com/
# mail in.. (AT) officeactivate (DOT) com for constructive criticism

User-agent: *
Disallow: /schuit
Disallow: /auction
Disallow: /Templates



Reply With Quote
  #2  
Old   
Jim Moe
 
Posts: n/a

Default Re: Is this robots.txt file doing what I think it is doing? - 03-19-2005 , 12:25 AM






w wrote:
Quote:
Someone please validate that this 'robots.txt' file that I put in the root,
allows all robots to look at my entire site, but not the 3 folders and it's
subfolders. Thank you.

I believe you must append a slash (/) to the end of directory paths.
You can validate your robots.txt at
<http://www.searchengineworld.com/cgi-bin/robotcheck.cgi>.

Quote:
User-agent: *
Disallow: /schuit
Disallow: /auction
Disallow: /Templates


--
jmm dash list (at) sohnen-moe (dot) com
(Remove .AXSPAMGN for email)


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.4
Copyright ©2000 - 2009, Jelsoft Enterprises Ltd.