Results 1 to 5 of 5
  1. #1
    Registered User
    Join Date
    Apr 2018
    Posts
    146

    Where robots.txt file uploaded to hosting directory?

    Hello friends,

    Where robots.txt file uploaded to hosting directory?

  2. #2
    Member
    Join Date
    May 2017
    Posts
    40
    Robots.txt is a text file located in the site's root directory that specifies for search engines' crawlers and spiders what website pages and files you want or don't want them to visit.

  3. #3

  4. #4
    Registered User
    Join Date
    Apr 2018
    Posts
    137
    Format and location rules for the robots.txt file:

    The file must be named robots.txt
    Your site can have only one robots.txt file.
    The robots.txt file must be located at the root of the website host that it applies to. For instance, to control crawling on all URLs below http://www.example.com/, the robots.txt file must be located at http://www.example.com/robots.txt. It cannot be placed in a subdirectory ( for example, at http://example.com/pages/robots.txt). If you're unsure about how to access your website root, or need permissions to do so, contact your web hosting service provider. If you can't access your website root, use an alternative blocking method such as meta tags.
    A robots.txt file can apply to subdomains (for example, http://website.example.com/robots.txt) or on non-standard ports (for example, http://example.com:8181/robots.txt).

  5. #5
    Senior Member
    Join Date
    May 2018
    Posts
    155
    If you are hoping that all crawlers and bots are going to respect what you are going to put on robots.txt, then you will be disappointed. Only a few of those many bots even care where you put robots.txt or what you put into

    But Google and other popular search engines are believed to respected a properly placed robots.txt ( in the root folder ). Good luck.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

  Find Web Hosting      
  Shared Web Hosting UNIX & Linux Web Hosting Windows Web Hosting Adult Web Hosting
  ASP ASP.NET Web Hosting Reseller Web Hosting VPS Web Hosting Managed Web Hosting
  Cloud Web Hosting Dedicated Server E-commerce Web Hosting Cheap Web Hosting


Premium Partners:


Visit forums.thewebhostbiz.com: to discuss the web hosting business, buy and sell websites and domain names, and discuss current web hosting tools and software.