Robot.txt File

[Shantanu]

Hello Friends

Can anyone tell me where to submit robot.txt file in our website ???

Please tell me

[georgeb]

Please look here (https://moz.com/learn/seo/robotstxt).

[Jagdish.P]

In the Root folder of your site. Make sure you have used correct syntax to block those segment of site which are of no use.

Thanks

[pharmasecure]

Keep in mind that only one file or folder can be used per Disallow line. You may add as many Disallow lines as you need. Once complete, save and upload your robots.txt file to the root directory of your site. For example, if your domain is www.mydomain.com, you will place the file at www.mydomain.com/robots.txt.

[MVMInfotech]

Simply type in your root domain, then add /robots.txt to the end of the URL. For instance, Moz's robots file is located at moz.com/robots.txt.

[stellasweety]

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.

[Rajdeep Bose]

Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web. By defining a few rules in this text file, you can instruct robots to not crawl and index certain files, directories within your site, or at all.

[iprism]

hen search engine crawlers come to your site it will be looking for some special file. i.e, call robots.txt file. This file tells to search engine spiders which pages of your site should be index and which pages of your site should be ignored.

[praveenitech40]

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.

[Prateektechnoso]

robots.txt file is giving instructions to the Crawler or Search Engine Spider to Crawl and Block Particular Website URLs.


For your information:

Prateektechnosoft is a Netsuite Partner and expertise in NetSuite ERP, CRM, Cloud CRM, PSA and other Netsuite Solutions. And also providing Netsuite services of implementation, integration, support & development services.

[RH-Calvin]

Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.

[fayeseom]

The robots.txt file, also known as the robots exclusion protocol or standard, is a text file that tells web robots (most often search engines) which pages on your site to crawl. It also tells web robots which pages not to crawl.

[roycpo]

Robots.txt is a text file that contain instructions for search engine spiders. The files lists pages to allow and disallow Google crawling. Robots.txt file helps to control the crawling of your website.