Which is better — Meta Robot tags or robots.txt?
Which is better — Meta Robot tags or robots.txt?
I think you should need to go with Robots.txt file, because it can compatibly with every server, there few chances like in other servers Meta robots code don't work.
robots.txt file is a publicly available file. Anyone can see what sections of your server you don't want robots to use. Meta tags are used to dis allow a single page from crawling. Meta Robots tag is much better as it helps in forcing the search engine crawlers not to index and display the hidden pages in your server.
|
Bookmarks