Printable View
what is robots.txt?
A robots.txt file tells search engine crawlers which URLs the crawler can access on your site.