Results 1 to 5 of 5
  1. #1
    Junior Member
    Join Date
    May 2019

    What is the limit of a robot.txt file?

    What is the limit of a robot.txt file?
    Bulk email service | Fastmailsender

  2. #2
    Senior Member
    Join Date
    Mar 2019
    Googlebot reads only the first 500kb present in the robots.txt file.

  3. #3
    Registered User
    Join Date
    Apr 2016
    Limit to robots.txt as far as filesize 500 KB as Vipin brings up. A document size might be authorized per crawler. Content which is after the most extreme file size might be overlooked. Google, as of now upholds a size utmost of 500kb.

  4. #4
    Senior Member
    Join Date
    Jun 2019
    Google currently enforces a size limit of 500kb

  5. #5
    Join Date
    Apr 2019
    Surat, India
    So, hope you get that final answer is "size limit of 500kb"

    So close the thread! the doubt is solved.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

  Find Web Hosting      
  Shared Web Hosting UNIX & Linux Web Hosting Windows Web Hosting Adult Web Hosting
  ASP ASP.NET Web Hosting Reseller Web Hosting VPS Web Hosting Managed Web Hosting
  Cloud Web Hosting Dedicated Server E-commerce Web Hosting Cheap Web Hosting

Premium Partners:

Visit to discuss the web hosting business, buy and sell websites and domain names, and discuss current web hosting tools and software.