PDA

View Full Version : What are Robots in SEO ?



Harry Mark
08-31-2012, 08:04 AM
Hello friends
Please give me the answer.

JanPaul
08-31-2012, 10:01 AM
I think this file name that controls the crawler to visit your that page of website which is under programing

vikramrandhawa
09-03-2012, 06:51 AM
Robots.txt is file which used to restrict website visit to search engine crawler.

ackerlee
09-03-2012, 07:39 AM
Robots are search engine crawler. They are useful for fetching the data from the website.

kevin1220
10-03-2012, 02:30 AM
Robots are search engine spider that crawls and indexes web pages.

jewelista
10-03-2012, 02:50 AM
Robots is a catch-all, or generic term for programs and automated scripts that “crawl” through the web (the Internet) and collect data from websites and anything else on the Internet that they can find.

blessy_smith
10-03-2012, 03:11 AM
I think Robot.txt is used to prevent crawlers crawl on a certain page...

sea queen
10-03-2012, 03:44 AM
Robots is illegal and it's search called search engine spider.

NolanBrian
10-03-2012, 04:24 AM
Very useful information..thanks

mariyajackson68
10-03-2012, 09:31 AM
when a particular site is placed inside robots.txt the site is nor crawled by the google search engine.

Engagedots
10-03-2012, 10:08 AM
It is to tell search robots which pages you would like them not to visit.

RayKersten
10-03-2012, 01:43 PM
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site (i.e. it is not a firewall, or a kind of password protection) and the fact that you put a robots.txt file is something like putting a note “Please, do not enter” on an unlocked door – e.g. you cannot prevent thieves from coming in but the good guys will not open to door and enter. That is why we say that if you have really sensitive data, it is too naive to rely on robots.txt to protect it from being indexed and displayed in search results.

generichard115
10-13-2012, 01:11 AM
Three-link way is a process that is used for online site optimization as it can widen the scope of links that can be directed to your own page.

Mithzi Rai
10-15-2019, 09:31 AM
Hello friends
Please give me the answer.

The robots.txt file, also known as the robots exclusion protocol or standard, is a text file that tells web robots (most often search engines) which pages on your site to crawl.

yuva12
10-15-2019, 10:47 AM
Robots.txt is a file that give instructions to the crawler as to which page to crawl and which page not to crawl.

davidweb09
10-22-2019, 03:20 AM
Robots is works as search engine indexing controller for your website. https://sydneyconveyancingonline.com/