How should i Prevent the URL from Getting Indexed by Google [Archive]

View Full Version : How should i Prevent the URL from Getting Indexed by Google

aanyajack

11-02-2023, 04:38 AM

How should i Prevent the URL from Getting Indexed by Google?

josiepete

11-02-2023, 11:19 AM

To prevent a URL from getting indexed by Google, you can use the following methods:

1. Robots.txt File: Create a robots.txt file and disallow indexing for the specific URL or directory you want to keep private. Add the following line to your robots.txt file:

User-agent: *
Disallow: /your-url

This tells web crawlers not to index the specified URL.

2. Meta Robots Noindex Tag: Add a <meta name="robots" content="noindex"> tag to the HTML <head> section of the specific web page you want to exclude from indexing. Google will respect this directive.

smithmary01

01-05-2024, 02:00 AM

There are two ways to prevent the url from indexing by google.
1. Robots.txt will tell the search engine crawlers that which specific directory or url should not be crawled.
2. Robots Meta Tag tell the search engine crawlers that the specific web page should not be crawled or indexed.

smartscraper

01-05-2024, 02:15 AM

Robots Meta Tag will be a better choice to block specific URLs/pages.