Results 1 to 7 of 7
  1. #1
    Senior Member
    Join Date
    Dec 2018
    Posts
    201

    Which is better Meta Robot tags or robots.txt?

    Which is better Meta Robot tags or robots.txt?

  2. #2
    Registered User
    Join Date
    Feb 2019
    Posts
    1,076
    Meta Robots tag is much better as it helps in forcing the search engine crawlers not to index and display the hidden pages in your server. The Meta tag option tells bots not to index individual files, whereas Robots.txt can be used to restrict access to entire directories. Both are supported by all crawlers.

  3. #3
    Registered User
    Join Date
    May 2019
    Location
    USA
    Posts
    149
    Use robots.txt :

    To stop engines from crawling certain sections of the site

    Robots Meta Tags:

    You can use a special HTML <META tag to tell robots not to index the content of a page, and/or not scan it for links to follow.

    <html>

    <head>

    <title>...</title>

    <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

    </head>

    There are two important considerations when using the robots <META> tag:

    robots can ignore your <META> tag. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.
    the NOFOLLOW directive only applies to links on this page. It's entirely likely that a robot might find the same links on some other page without a NOFOLLOW (perhaps on some other site), and so still arrives at your undesired page.

  4. #4
    Junior Member
    Join Date
    Jun 2019
    Posts
    20
    Meta Robot tags are better as it helps your website to rank in google pages

  5. #5
    Member
    Join Date
    May 2019
    Posts
    95
    Meta Robot tags are better than robots.txt

  6. #6
    Senior Member
    Join Date
    Nov 2018
    Posts
    457
    The Robot Meta tag option is used to tell the bots not to index any individual files, whereas Robots.txt can be used to restrict access to entire directories as well.

  7. #7
    Member
    Join Date
    Jun 2019
    Posts
    66
    The Meta tag option tells bots not to index individual files, whereas Robots.txt can be used to restrict access to entire directories. Both are supported by all crawlers

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

  Find Web Hosting      
  Shared Web Hosting UNIX & Linux Web Hosting Windows Web Hosting Adult Web Hosting
  ASP ASP.NET Web Hosting Reseller Web Hosting VPS Web Hosting Managed Web Hosting
  Cloud Web Hosting Dedicated Server E-commerce Web Hosting Cheap Web Hosting


Premium Partners:


Visit forums.thewebhostbiz.com: to discuss the web hosting business, buy and sell websites and domain names, and discuss current web hosting tools and software.