What’s the significance of Robots.txt file in a website?

[loveblackmagic]

Hlo Friends,
Can anyone tell me, What’s the significance of Robots.txt file in a website?

[vinodkumar]

Robots.txt file is most important file in seo for a website. Whenever crawler crawl a website it search for robots file to check any instructions.

[Powerfulvasikar]

Robots.text file tell the search engine which file is indexed or which is not.

[jane1]

Robot.txt file direct search engine to crawl a web site or not.

[sinelogixweb]

The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.

[eramaker]

The robots.txt is a normal text file placed on your web server which tells Google bot if they should access your file or not.

[chiragpatel108]

Robots.txt file is important because you can easily crawl web page by giving permission to search engine.

[roycpo]

Robots.txt file helps a website to index on the search engine and it continuously crawls the different data for that website.

[daikaads]

Well, robots.txt is a file which is used to inform the search engines about the crawling and indexing of the web page or website. You can block or allow the search engine bots to crawl your website.

[sadianisar]

robots.txt is a file in which we instruct web crawler to not visit and index the link or page or website. It is an essential On-Page SEO element used for newly built websites.

[Maitri]

Robot.txt is a file which is actually a text file created to instruct robots (search engine robots) that how and which pages to crawl of a website.

[SidharthNaraya]

It like as file permission access