what is robot text?
what is robot text?
it gives instruction to search engine spider that which pages of your sites you want to index and which pages want to ignore, there are many pages in website but not all useful for users so it's not important to index those pages in search engine, A robots.txt file tell to the robots which pages or parts of the website that should be index and which parts should not.
an example of Robots.txt file is below:
(!) User-agent: Googlebot/MSNbot/yahoobot.
Disallow: add your link here which do not want to index.
(!!) User-agent:*
Disallow: /
(!!!)(*) indicates all the search engines.
(/) indicates all pages of a site.
Hi I am Bmarl from United States and I work with one of the most reputed company named - Ancientwriters who provide the perfect writing as well as publishing services . I have been working there from past few years. If you are looking for such writing and publishing services then you can visit their website.
Robots.txt is a text file webmasters create to instruct web robots (typically search engine robots) how to crawl pages on their website.
robots.txt is a text file its helps to block all web crawler from all web pages or specific web crawler from specific web pages.
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.
Robots.txt is a file which is used by webmasters to instruct search engine robots not to crawl the web pages in the website.
Robots.txt file help to control your site indexing by Google, Bing & Yahoo.
Business | Pinterest | $99 SEO | Shopping | Society | Health | Travel | Home | Pearltrees | Flipboard | Business Services | Real Estate | Art | Small Business SEO | SEO Company | Contently | Mix | Plurk
Robot.txt is an on-page SEO technique and it is basically used to allow for the web robots also known as the web wanderers, crawlers or spiders. It is a program that traverses the website automatically and this helps the popular search engine like Google to index the website and its content.
Robots.txt is a special kind of file that is not HTML or any other type. The purpose of having a robots.txt file is to tell the search engine that what content on your site is allowing search bots and indexes.
robots.txt is text file. it helps to block all web crawler from all pages or specific web crawler form specific web page.
Robots.txt is a text file webmasters create to instruct web robots (typically search engine robots) how to crawl pages on their website.
Online Free Recharge | Online mobile recharge | aircel online recharge | Vodafone online recharge | online bill payment | airtel postpaid bill payment | Online DTH recharge | Tata sky recharge | dish TV recharge | airtel DTH recharge | Online datacard recharge | online gas bill payment |Online electricity bill payment | Online light bill payment | online bus booking | online bus ticket booking | Book hotel Online | Book flight tickets
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots.
It is Robot txt and this means that there are certain places on the website where there is personal information and in these places customers are not allowed to go so in this case robot txt is used
You can refer it - http://forums.hostsearch.com/showthr...-is-Robots-txt
Server Management Company
India's Leading Managed Service Provider | Skype: techs24x7
Cpanel Technical Discussions - Lets talk !
|
Bookmarks