PDA

View Full Version : Robot text.



terrijhon
01-23-2013, 03:13 AM
What Is robot text and what its function in the website or SEO. Please give me its answers’ am waiting for its answers.

arindamdutta16
01-23-2013, 04:26 AM
The robots.txt is a text file in your web site that informs search engine bots how to crawl and index website or web pages. It is great when search engines frequently visit your site and index your content but often there are cases when indexing parts of your online content is not what you want.

Hire Developer
01-23-2013, 05:28 AM
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.

thearav
01-23-2013, 07:10 AM
Robots.txt is a file which is used to instruct the google that not to read the pages which the file has.

blessy_smith
01-24-2013, 12:41 AM
Robots.txt is a file through which you can guide search engines to crawl or not to crawl certain sections of your website.
Google specifically follows instructions given in this robots.txt file

webcreations
01-24-2013, 01:22 AM
The robots.txt is a simple text file in your web site that inform search engine bots how to crawl and index website or web pages.By default search engine bots crawl everything possible unless they are forbidden from doing so. They always scan the robots.txt file before crawling the web site.The robots.txt is usually placed in the root folder of your web site.using the robots.txt file, you can hide the pages such as user profiles and other temp folders from being indexed and does not divulge your SEO effort into junk or the pages which are useless for the search results. In general, you results will be more precise and better valued.

theavi
01-24-2013, 01:24 AM
Robot.txt file is used to tell google to kindly do not read the pages which txt file has. It is used to do not crawl some pages in website.

ackerlee
01-24-2013, 01:28 AM
Robots.txt is a file through which you can guide search engines to crawl or not to crawl certain sections of your website. Google specifically follows instructions given in this robots.txt file.It is used to prevent crawlers crawl on a certain page...

jamsen
01-24-2013, 09:53 AM
The function of robots.txt :
- allowing and dissalowing site for being crawl by spider
- for letting spider know our sitemap, ex: for google spider to know our sitemap

JerrickYeoh
01-24-2013, 08:54 PM
Robots.txt is common name of a text file that is uploaded to a Web site's root directory and linked in the html code of the Web site. The robots.txt file is used to provide instructions about the Web site to Web robots and spiders. Web authors can use robots.txt to keep cooperating Web robots from accessing all or parts of a Web site that you want to keep private.

jaysh4922
01-24-2013, 11:39 PM
Robot.txt is an on-page SEO technique and it is basically used to allow for the web robots also known as the web wanderers, crawlers or spiders. It is a program that traverses the website automatically and this helps the popular search engine like Google to index the website and its content.