What is robots.txt and how search engine behave with it?

[PeteMrcus90]

Hello all, please define robots.txt.
Also I have come to know that robots.txt is best to use for search engine better crawling and indexing. Please suggest, all replies would be highly appreciated.
Thanks

[Vamp1re]

Robots text file are those which forced to search engine how to crawl and index the your site or web pages.
It is great when search engines frequently visit your site and index your content but often there are cases when indexing parts of your online content is not what you want.

[ShivJaiswaL]

hello Pete, robots.txt is a text file resides in root DIR of host website. Almost all major search engines like Google, Bing, http://www.oko69.com ...... respect the presence of robots.txt. It permits us to block complete or partial website portion. Make Sure to use a correct syntax of robots.txt. Additionally, validating a robots.txt can be a nice feature to use it properly .
thanks

[blessy_smith]

Robots.txt is a file through which you can guide search engines to crawl or not to crawl certain sections of your website.