Anyone explains to me What? If Robot.txt tester blocked a website or webpage?
Results 1 to 4 of 4
Thread: What? If Robot.txt blocked?
-
02-22-2022, 07:58 PM #1Registered User
- Join Date
- Mar 2020
- Posts
- 781
What? If Robot.txt blocked?
-
02-23-2022, 12:40 AM #2Registered User
- Join Date
- Jan 2022
- Location
- United States
- Posts
- 38
If your web page is blocked with a robots. txt file, its URL can still appear in search results, but the search result will not have a description. Image files, video files, PDFs, and other non-HTML files will be excluded.
-
02-23-2022, 01:07 AM #3Registered User
- Join Date
- Dec 2021
- Location
- USA
- Posts
- 29
In this blocked robots.txt case URL can still appear in search engine results, but the search result will not have a description. Image files, video files, PDFs, and other non-HTML files will be excluded. If you see this search result for your page and want to fix it, remove the robots.
-
02-23-2022, 01:18 AM #4Senior Member
- Join Date
- Sep 2021
- Location
- Bangalore, India
- Posts
- 437
'Indexed, though blocked by robots. txt' indicates that Google has found your page, but has also found an instruction to ignore it in your robots file
Reply With Quote
Posting Permissions
|
Bookmarks