Results 1 to 3 of 3
  1. #1
    Registered User
    Join Date
    Aug 2018
    Posts
    1,141

  2. #2
    Senior Member
    Join Date
    Nov 2021
    Location
    Bangalore
    Posts
    352
    No, a robots. txt file is not required for a website. If a bot comes to your website and it doesn't have one, it will just crawl your website and index pages as it normally would.

  3. #3
    Senior Member
    Join Date
    Sep 2021
    Location
    India
    Posts
    464
    A robots.txt file is a plain text file that specifies whether or not a crawler should or shouldn't access specific folders

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

  Find Web Hosting      
  Shared Web Hosting UNIX & Linux Web Hosting Windows Web Hosting Adult Web Hosting
  ASP ASP.NET Web Hosting Reseller Web Hosting VPS Web Hosting Managed Web Hosting
  Cloud Web Hosting Dedicated Server E-commerce Web Hosting Cheap Web Hosting


Premium Partners:


Visit forums.thewebhostbiz.com: to discuss the web hosting business, buy and sell websites and domain names, and discuss current web hosting tools and software.