What is the limit of a robot.txt file?
Results 1 to 5 of 5
-
06-25-2019, 07:26 AM #1Junior Member
- Join Date
- May 2019
- Posts
- 21
What is the limit of a robot.txt file?
Bulk email service | Fastmailsender
-
06-25-2019, 07:39 AM #2Senior Member
- Join Date
- Mar 2019
- Posts
- 131
Googlebot reads only the first 500kb present in the robots.txt file.
-
06-25-2019, 07:51 AM #3Registered User
- Join Date
- Apr 2016
- Posts
- 792
Limit to robots.txt as far as filesize 500 KB as Vipin brings up. A document size might be authorized per crawler. Content which is after the most extreme file size might be overlooked. Google, as of now upholds a size utmost of 500kb.
-
06-26-2019, 12:57 AM #4Senior Member
- Join Date
- Jun 2019
- Posts
- 206
Google currently enforces a size limit of 500kb
-
06-26-2019, 01:39 AM #5Member
- Join Date
- Apr 2019
- Location
- Surat, India
- Posts
- 77
So, hope you get that final answer is "size limit of 500kb"
So close the thread! the doubt is solved.
Reply With Quote
Posting Permissions
|
Bookmarks