This week’s Design and Usability Tactics e-newsletter explains how you can use a robots.txt file to control which portions of your site the search engine robots will index.
How do you control what search engine robots index? Does the method suggested in the column sound like something you’ll try?
In the column, the author mentions that syntax errors aren?t always obvious to humans; however, you can use online validators to check your robots.txt file. Here are URLs to three online validators that you can use:
http://www.searchengineworld.com/cgi-bin/robotcheck.cgi
http://www.designerwiz.com/test/robots.txt_validator.htm
http://www.sxw.org.uk/computing/robots/check.html
* Please delete any extra spaces that may appear when you paste these URLs into your browser.
If you haven’t subscribed to the Design and Usability Tactics TechMail, visit our subcenter to subscribe to this free e-newsletter today:
http://nl.com.com/acct_mgmt.jsp?brand=builder