Web Design Blog
What is Robots.txt and why should I use it?
July 31st, 2007 by Dustin BrewerI am here to answer just that question. There has been a lot of mystery I have gathered from fellow web developers about the benefits of using a robots.txt. Most are under the false impression that the only purpose for this text file is to tell search robots not to index a certain section of their site. This isn’t the case at all! There are much more useful reasons to be using a Robots.txt that have to do very much with Search Engine Optimization (SEO) that most people don’t realize. Read more »
-
About Dustin Brewer
Dustin Brewer is a freelance web designer based out of Oklahoma City, OK passionate about web standards, and beautiful web design. Dustin Brewer has been in the web design industry for over 8 years through freelance and professional experience. If you are interested in hiring Dustin Brewer please visit the web design services page to find out more information. You can also checkout my web design portfolio.
Contact Information
Contact, interact or keep in touch through any or all of the methods listed below.