Robot.txt file give instruction to google crawler which file not to index or crawl.