Is Robots allow or not to all Search Engines ?

[Fadia Sheetal]

Hay Users,

I start working on a Website and bit confuse about the robots. What is the right extension for search engines allow or not to crawl the website ?