Page 1 of 2 12 LastLast
Results 1 to 15 of 17
  1. #1
    Senior Member
    Join Date
    Aug 2017
    Location
    India
    Posts
    128

    Which is better Robots.text or Meta Robots Tag?

    Which is better Robots.text or Meta Robots Tag?

  2. #2
    Registered User
    Join Date
    Feb 2017
    Posts
    754
    I prefer Meta Robots Tag as it is easier to implement on the website and to block the search engines.

  3. #3
    Member
    Join Date
    Aug 2017
    Posts
    54
    Robots meta tag gives option that crawl or not crawl specific page.It also informs about do-follow and no-follow links on the web page.whereas robots.txt is a list of a directories which mainly used for restriction about crawling of a specific web pages.so both used for different purposes and not better than each other.

  4. #4
    Senior Member
    Join Date
    Jun 2013
    Location
    Forum
    Posts
    5,019
    Robots.txt file is better compared to meta robots tag. Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
    Cheap VPS | $1 VPS Hosting
    Windows VPS Hosting | Windows with Remote Desktop
    Cheap Dedicated Server | Free IPMI Setup

  5. #5
    Registered User
    Join Date
    Aug 2017
    Posts
    1
    Meta tag can be used to disallow a single page. ... There is the very huge difference between meta robot and robots.txt. In robots.txt, we ask crawlers which page you have to crawl and which one you have to exclude but we don't ask crawler to not to index those excluded pages from crawling.

  6. #6
    Senior Member deepakrajput's Avatar
    Join Date
    Feb 2012
    Location
    CA
    Posts
    1,223
    Robots.txt is much better than others.

  7. #7
    Senior Member
    Join Date
    Jul 2017
    Location
    Surat
    Posts
    195
    Robots.txt is an actual text file, whereas meta robot tags are meta directives. Robots.txt dictates site or directory-wide crawl behavior, whereas meta robots can dictate indexation behavior at the individual page (or page element) level.

  8. #8

  9. #9

  10. #10
    Member
    Join Date
    Aug 2017
    Location
    India
    Posts
    86
    Quote Originally Posted by seojesica View Post
    Robots.txt file is very important, but you can use both, because both have own importance.
    Yes, Why can't we use the both methods because they both do the same work ..

  11. #11
    Member
    Join Date
    Jul 2017
    Posts
    49
    Often the robots.txt file located at the root of the domain name is the ideal tool for communicating with search engines (Robots with Google, Yahoo and Microsofts) in indexing websites. However, in some cases, the /robots.txt file becomes limited and its use is very cumbersome. Heavy META "robots" at the moment are useful even when using it. Quite different from robots.txt file

  12. #12
    Member
    Join Date
    Jul 2017
    Posts
    53
    Robots' meta tag is important

    Robots.txt file is a text file for restricting bots (robots, search engine crawlers ) from a website or certain pages on the website. Using a robots.txt file and with a disallow direction, we can restrict bots or search engine crawling program from websites and or from certain folders and files.

    A robots meta tag is used to give direction for robots or the spiders regarding crawling and indexing of particular page. By using a robots meta tag which requests bots to avoid crawling and indexing, that page can be avoided from crawling by search engine bots or robots and avoid indexing in search results. A request name “follow” or “nofollow” is used in robots meta tags provided in the head section of a webpage to consider or not consider the counting of links provided on the page. Not adding a robots meta tag will be considered as the default direction such as index, follow.

  13. #13
    Senior Member
    Join Date
    Mar 2017
    Posts
    422
    Use robots.txt file, use meta robots when the robots.txt file does not work properly because meta robots hurt sites sometimes.
    If you use the robots.txt file then you can block different directories or pages with one file, if you use meta robots you have to add it on every page where you want to be and indirectly it makes your code lengthy.

  14. #14
    Senior Member
    Join Date
    Aug 2017
    Location
    West Patel Nagar, Delhi
    Posts
    205
    robots.txt file is better and easy to use.

  15. #15
    Registered User
    Join Date
    Nov 2013
    Posts
    905
    There is a very huge difference between

    Robots.txt: we ask crawlers which page you have to crawl

    Meta Tag: which one you have to exclude but we don't ask crawler to not to index those excluded pages from crawling.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

  Find Web Hosting      
  Shared Web Hosting UNIX & Linux Web Hosting Windows Web Hosting Adult Web Hosting
  ASP ASP.NET Web Hosting Reseller Web Hosting VPS Web Hosting Managed Web Hosting
  Cloud Web Hosting Dedicated Server E-commerce Web Hosting Cheap Web Hosting


Premium Partners:


Visit forums.thewebhostbiz.com: to discuss the web hosting business, buy and sell websites and domain names, and discuss current web hosting tools and software.