What is the robots.txt file?

Please can anyone explain me what is robots.txt file and its correct format?

Robots.txt a text file made on notepad which instructs search engines about their allowed crawling or visitng areas on full website. Example content for a robots.txt file which give full permission to all search engines to crawl full website is given below:
User-agent: *
Disallow:

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site (i.e. it is not a firewall, or a kind of password protection) and the fact that you put a robots.txt file is something like putting a note “Please, do not enter” on an unlocked door – e.g. you cannot prevent thieves from coming in but the good guys will not open to door and enter. That is why we say that if you have really sen sitive data, it is too naïve to rely on robots.txt to protect it from being indexed and displayed in search results.

The concept and structure of robots.txt has been developed more than a decade ago and if you are interested to learn more about it, visit http://www.robotstxt.org/ or you can go straight to the Standard for Robot Exclusion because in this article we will deal only with the most important aspects of a robots.txt file. Next we will continue with the structure a robots.txt file.

Actually this term design for making any big changes in your current well developed site.Some people want some changes without any seo risk......

Roberts.text file is the file which is used to hide the privacy page or any private page from the Google spider.So that we maintain our privacy.

Robots.txt is one kind of file that is mainly used to tell or instruct search engine robot regarding your website so that frequently visit your website and index your content. Each webpage must have different robots.txt file just like http://www.abc.com/robots.txt .
Example:-
User-Agent: *
Disallow:
Hence, it visits all pages because * contains all robots

thanks for shearing your experiences with us..

The Robot Exclusion Standard, also known as the Robots Exclusion Protocol or robots.txt protocol.It is a convention to prevent cooperating web crawlers and other web robots from accessing all or part of a website which is otherwise publicly viewable. These are often used by search engines to categorize and archive web sites, or by webmasters to proofread source code.

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site (i.e. it is not a firewall, or a kind of password protection) and the fact that you put a robots.txt file is something like putting a note “Please, do not enter” on an unlocked door – e.g. you cannot prevent thieves from coming in but the good guys will not open to door and enter. That is why we say that if you have really sen sitive data, it is too naïve to rely on robots.txt to protect it from being indexed and displayed in search results.

Robot.txt is an on-page SEO technique and it is basically used to allow for the web robots also known as the web wanderers, crawlers or spiders. It is a program that traverses the website automatically and this helps the popular search engine like Google to index the website and its content.

Robot.txt file is using for disallow duplicate page or images from crawls.

Robots.txt is a file through which you can guide search engines to crawl or not to crawl certain sections of your website.

Robots.txt file is used to what pages you want crawler to crawl or not.

The Robots.txt file of a website will work when it is used as a request to specific robots to ignore directories or files specified within the Robots.txt file.