What is a crawler?

[jacobharrison76]

What is a crawler?

[annabelle123]

several crawler there, what crawler you ask.

[rajivdevkk]

crawlers is search engines spiders. crawlers main work to crawls the pages and save its to data base to show search engine results.

[josephinek]

It is also called as web crawlers.
It is a program.
It helps search engines use to crawl web pages for indexing.

[drwebsites]

a program that systematically browses the World Wide Web in order to create an index of data for the search engine.

[RH-Calvin]

Crawlers are programmable bots that move through your webpage source code and identify your webpage content. They help to cache and index your webpages in search engines.

[onlinebigboss]

crawler is the Google spider and it is also crawl your websites and store your information on data base

[NicholasK]

A Web crawler is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing. A Web crawler may also be called a Web spider, an ant, an automatic indexer, or (in the FOAF software context) a Web scutter.

Web search engines and some other sites use Web crawling or spidering software to update their web content or indexes of others sites' web content. Web crawlers can copy all the pages they visit for later processing by a search engine that indexes the downloaded pages so that users can search them much more quickly.

[yepionline]

A crawler is a spider (robots) that visit Web sites and reads their pages and other information in order to create entries for a search engine index.

[jhonsmith123]

Hi friends,

A program that systematically browses the World Wide Web in order to create an index of data.

[Sophiacalvin]

web crawler is the spider that can helps in accessing many web pages in one crawler.

[naveenkumar47]

great information shared in this thread thanx

[rohit]

Crawlers and bots are more or less the same they are used to automate the work which is repetitive.

Crawlers are used by search engines to make a copy of the webpage in their database, consider a google bot / crawler visits your page then google will be having a copy of your webpage and analyze the page and rank them accordingly, Also they make a list of links available in your website and form a hierarchical order of webpages available, making one page as home and others as child pages. This type of crawling makes the search engines check for new pages available, so it can update its database.

[Ezio]

A crawler is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index.

[marclewis]

It is simply a program that browses the world wide web in order to provide up to date to the particular search engine.