robots.txt

[jackthomas087]

Reboot Txt File .

The /robots.txt is a de-facto standard, and is not owned by any standards body. There are two historical descriptions:

the original 1994 A Standard for Robot Exclusion document.
a 1997 Internet Draft specification A Method for Web Robots Control..

[arianagrand]

Thank for the sharing useful information ..

[Teamwork]

robots.txt file is a part of seo.

[jaysh4922]

Robot.txt is an on-page SEO technique and it is basically used to allow for the web robots also known as the web wanderers, crawlers or spiders. It is a program that traverses the website automatically and this helps the popular search engine like Google to index the website and its content.

[Raju]

Robots.txt is used to gives search engines information about the pages a company wants indexed or crawled. You can find this page by doing to YOURDOMAIN/robots.txt.

[amit45]

Hello Friends,
RObot.txt file is a file which save in root directory. Robot.txt tells search engine engine crawler that which pages has to be index or not?

[ashiselma]

Robots.txt files that help ensure Google and other search engines are crawling and indexing your site properly.

[StuartSpindlow1]

You can block a particular page of your website with robot.txt which you don't want to crawl by Google.