Do I need robots.txt?

[juliaalan]

Do I need robots.txt?


Oryon Networks (http://www.oryon.net) | Singapore Web Hosting (http://www.oryon.net) | Best web hosting provider (http://www.oryon.net) | Best web hosting in SG (http://www.oryon.net) | Oryon SG (https://blog.oryon.net/)

[donamiller90]

No, a robots. txt file is not required for a website. If a bot comes to your website and it doesn't have one, it will just crawl your website and index pages as it normally would.

[Cyril30]

A robots.txt file is a plain text file that specifies whether or not a crawler should or shouldn't access specific folders