HostGator.com » Support Portal

How to use robots.txt

What is the purpose of the robots file?

This file tells search engines what they should and should not index (make available as links).

You can simply create a blank file and name it robots.txt. (This will reduce site errors and allow all search engines to rank you if they want.)

If you want to stop search engines from ranking you, use this code:
#Code to not allow any search engines!
User-agent: *
Disallow: /
See more codes here:
http://www.robotstxt.org/robotstxt.html

If you have read our knowledge base and still have questions or need more information, please contact us via phone or live chat for immediate assistance.

Post Comment