PDA

View Full Version : How to work Google Crawler?



JamessLee
09-25-2012, 12:31 PM
Hello My Friends,

I Would like to know that how is the Google Crawler work?

Regards,

James Lee.

blessy_smith
09-26-2012, 01:50 AM
Googlebot is Google’s web crawling robot, which finds and retrieves pages on the web and hands them off to the Google indexer. It’s easy to imagine Googlebot as a little spider scurrying across the strands of cyberspace, but in reality Googlebot doesn’t traverse the web at all. It functions much like your web browser, by sending a request to a web server for a web page, downloading the entire page, then handing it off to Google’s indexer.

rosesmark
09-26-2012, 02:30 AM
I got one link which described the full detail about Google Crawler. I am hoping that you get your answer through this link. Which is mention below.
http://www.googleguide.com/google_works.html

fivebucksdeals
09-26-2012, 03:06 AM
Googlebot is Google’s web crawling robot, which finds and retrieves pages on the web and hands them off to the Google indexer. It’s easy to imagine Googlebot as a little spider scurrying across the strands of cyberspace, but in reality Googlebot doesn’t traverse the web at all. It functions much like your web browser, by sending a request to a web server for a web page, downloading the entire page, then handing it off to Google’s indexer.

Googlebot consists of many computers requesting and fetching pages much more quickly than you can with your web browser. In fact, Googlebot can request thousands of different pages simultaneously. To avoid overwhelming web servers, or crowding out requests from human users, Googlebot deliberately makes requests of each individual web server more slowly than it’s capable of doing.

abhishekabhi
09-26-2012, 08:37 AM
HI

soryy I have no idea.

abhishekabhi
09-26-2012, 08:39 AM
what is top five directories List????????????

Please help me.

govindsharan
09-26-2012, 08:53 AM
The Google crawler, known as Googlebot, which finds and retrieves pages on the web and hands them off to the Google indexer. These pages are stored in Google’s index database.:D

Engagedots
09-26-2012, 09:16 AM
Googlebot, a web crawler that finds and fetches web pages.

seocrat
09-26-2012, 09:23 AM
Googlebot is Google’s web crawling robot, which finds and retrieves pages on the web and hands them off to the Google indexer. It’s easy to imagine Googlebot as a little spider scurrying across the strands of cyberspace, but in reality Googlebot doesn’t traverse the web at all. It functions much like your web browser, by sending a request to a web server for a web page, downloading the entire page, then handing it off to Google’s indexer.

Thanks for the information, its really helpful.

Thanks,

Allison7
09-27-2012, 09:50 AM
Googlebot is Google’s web crawling robot, which finds and retrieves pages on the web and hands them off to the Google indexer. It’s easy to imagine Googlebot as a little spider scurrying across the strands of cyberspace, but in reality Googlebot doesn’t traverse the web at all. It functions much like your web browser, by sending a request to a web server for a web page, downloading the entire page, then handing it off to Google’s indexer.

Googlebot consists of many computers requesting and fetching pages much more quickly than you can with your web browser. In fact, Googlebot can request thousands of different pages simultaneously. To avoid overwhelming web servers, or crowding out requests from human users, Googlebot deliberately makes requests of each individual web server more slowly than it’s capable of doing.

jamsen
09-29-2012, 09:41 AM
"Crawler" is a generic term for any program (such as a robot or spider) used to automatically discover and scan websites by following links from one webpage to another. Google's main crawler is called Googlebot.

Google's spiders regularly crawl the web to rebuild our index. Crawls are based on many factors such as PageRank, links to a page, and crawling constraints such as the number of parameters in a URL. Any number of factors can affect the crawl frequency of individual sites.