PDA

View Full Version : Which is better — Meta Robot tags or robots.txt?


shoppingswag
07-09-2019, 02:48 AM
Which is better — Meta Robot tags or robots.txt?
eCommerceChamp
07-09-2019, 05:03 AM
Use robots.txt :

To stop engines from crawling certain sections of the site

Robots Meta Tags:

You can use a special HTML <META tag to tell robots not to index the content of a page, and/or not scan it for links to follow.

<html>

<head>

<title>...</title>

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

</head>

There are two important considerations when using the robots <META> tag:

robots can ignore your <META> tag. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.
the NOFOLLOW directive only applies to links on this page. It's entirely likely that a robot might find the same links on some other page without a NOFOLLOW (perhaps on some other site), and so still arrives at your undesired page.
bhavesh_11
07-09-2019, 05:11 AM
Meta Robot tags are better as it helps your website to rank in google pages
NoahKlindon
07-09-2019, 07:07 AM
Meta Robot tags are better than robots.txt
Saravanan28
07-09-2019, 09:20 AM
The Robot Meta tag option is used to tell the bots not to index any individual files, whereas Robots.txt can be used to restrict access to entire directories as well.
HeavenlyBorn
07-11-2019, 12:55 AM
The Meta tag option tells bots not to index individual files, whereas Robots.txt can be used to restrict access to entire directories. Both are supported by all crawlers