Is the robots.txt file necessary for seo?

[mackhunt88]

I know that in the robots.txt file i can tell google what pages on my site not to crawl. Are there any other uses of this file? Can it be useful for seo in any way?

[ChrisDevlin]

For every website Robots.txt file is necessary. In that file only we need to give commands for all search engine bots for crawling purpose. A perfect definition from Wikipedia about Robots.txt.

"The Robot Exclusion Standard, also known as the Robots Exclusion Protocol or robots.txt protocol, is a convention to prevent cooperating web crawlers and other web robots from accessing all or part of a website which is otherwise publicly viewable."

[Amy.Sarin]

Other than that, it also helps Search Engines to locate the Sitemap of the website and hence crawl the entire website in depth... helping in your rankings and traffic.

[lingoway]

Robots.txt file is necessary for every website. Upload this file on server, it doesn't matter whether it is empty.

[marc1]

Robot.txt is very necessary file for any website. Google crawler is always supposed to read it.

[watson123]

Yes,Robots.txt file is very important to ensure effective Search Engine Optimization of a website.

[ajaykrgc]

hi,
i am agree with your answer

[smitholivia]

Yes it is good practice having robots.txt on a website despite if you dont want to restrict crawler to any web page or folder.

[blessy_smith]

Google crawler suppose to read the robot.txt file and it is very important for your website.

[fashion_today]

Yes Robot.txt is very important for the website crawl ....

[katherineeobrya]

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.

[pestindia86]

hi,
yes robots.txt is important for SEO.

[sabrinasai]

You can block the secured and confidential pages through robots.txt file & prevent search engines from crawling those pages

[prymagestore]

Yes, it is necessary in SEO because when search engine crawler comes to your site, first it look for a file that robots.txt. Robots.txt is a text file which instructs Search Engine robots what to index on your site and what to be unindexed.