Robot.Text File

[armor143]

What is robot.text file?

"Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web. By defining a few rules in this text file, you can instruct robots to not crawl and index certain files, directories within your site, or at all.The location of robots.txt is very important. It must be in the main directory because otherwise user will not be able to find it – they do not search the whole site for a file named robots.txt.

[ilmkidunya1]

hmmm nice information thanks for sharing with us

[JohnRosatti]

Robot.txt file is used to make the pages or content private from Google crawlers.Sometimes we add a code or page on website for visitors and Would not like that Google crawls them .So The robot.txt files Full-fill the requirement.

[katherineeobrya]

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.

[sabrinasai]

"Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web. By defining a few rules in this text file, you can instruct robots to not crawl and index certain files, directories within your site, or at all. For example, you may not want Google to crawl the /images directory of your site, as it's both meaningless to you and a waste of your site's bandwidth. "Robots.txt" lets you tell Google just that.

[watson123]

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site (i.e. it is not a firewall, or a kind of password protection) and the fact that you put a robots.txt file is something like putting a note “Please, do not enter” on an unlocked door – e.g. you cannot prevent thieves from coming in but the good guys will not open to door and enter. That is why we say that if you have really sen sitive data, it is too naïve to rely on robots.txt to protect it from being indexed and displayed in search results.

[Openxcell Inc]

Robot.txt is a text file which contains the instruction for the search engines crawlers that which links to be crawl and index by search engines.

[armor143]

Robots.txt is a text file not a html file you put on your site to tell search robots which pages you would like them not to visit.It is by no means mandatory for search engines but generally search engines obey what they are asked not to do. whether it's Google trying to index the entire web, or a spam bot collecting any email address it could find for less than honorable intentions. As site owners, what little control we have over what robots are allowed to do when they visit our sites exist in a magical little file called "robots.txt."

[ilmkidunya1]

it helps you to for indexing the pages you want or not

[marc1]

Robot.txt file mostly use to hide any page or information from public and some times from Google also.

[synapsesite]

What is robot.text file?

"Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web...............


Love that line. Nice info. :)

[abhishekabhi]

Robot .txt file is contains instruction of the search engine.

[rajjan011]

I really don't know how to upload robots.txt file.I also want to know more about this.