Page 1 of 2 12 LastLast
Results 1 to 15 of 17
  1. #1

  2. #2

  3. #3
    Senior Member
    Join Date
    Dec 2019
    Posts
    1,580
    A web crawler, or spider, is a type of bot that's typically operated by search engines like Google and Bing. Their purpose is to index the content of websites all across the Internet so that those websites can appear in search engine results.

  4. #4
    Senior Member
    Join Date
    Sep 2020
    Location
    Maxico
    Posts
    282
    Search engines work by crawling hundreds of billions of pages using their own web crawlers. These web crawlers are commonly referred to as search engine bots or spiders. A search engine navigates the web by downloading web pages and following links on these pages to discover new pages that have been made available.

  5. #5

  6. #6
    Member
    Join Date
    Feb 2021
    Location
    Europe
    Posts
    80
    Web crawler, or spider, is a type of bot that's typically operated by search engines like Google and Bing. Their purpose is to index the content of websites all across the Internet so that those websites can appear in search engine results.

  7. #7

  8. #8
    Senior Member
    Join Date
    Mar 2020
    Posts
    1,214
    A web crawler, or spider, is a type of bot that's typically operated by search engines like Google and Bing. Their purpose is to index the content of websites all across the Internet so that those websites can appear in search engine results.

  9. #9
    Senior Member dennis123's Avatar
    Join Date
    Apr 2013
    Location
    Bangalore
    Posts
    3,480
    Crawling is the process by which Googlebot visits new and updated pages to be added to the Google index. We use a huge set of computers to fetch (or "crawl") billions of pages on the web. The program that does the fetching is called Googlebot (also known as a robot, bot, or spider).

  10. #10
    Senior Member
    Join Date
    Sep 2019
    Posts
    765
    A crawler is a computer program that automatically searches documents on the Web. Crawlers are primarily programmed for repetitive actions so that browsing is automated. Search engines use crawlers most frequently to browse the internet and build an index.

  11. #11
    Registered User
    Join Date
    Jul 2019
    Posts
    35
    Quote Originally Posted by GeethaN View Post
    A web crawler, or spider, is a type of bot that's typically operated by search engines like Google and Bing. Their purpose is to index the content of websites all across the Internet so that those websites can appear in search engine results.
    Yes, This is a good and perfect answer.

  12. #12
    Senior Member
    Join Date
    Dec 2019
    Posts
    1,580
    A web crawler, or spider, is a type of bot that's typically operated by search engines like Google and Bing. Their purpose is to index the content of websites all across the Internet so that those websites can appear in search engine results.

  13. #13

  14. #14
    Registered User
    Join Date
    Nov 2020
    Location
    France
    Posts
    312
    A crawler is a computer program that automatically searches documents on the Web. Crawlers are primarily programmed for repetitive actions so that browsing is automated. Search engines use crawlers most frequently to browse the internet and build an index.

  15. #15

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

  Find Web Hosting      
  Shared Web Hosting UNIX & Linux Web Hosting Windows Web Hosting Adult Web Hosting
  ASP ASP.NET Web Hosting Reseller Web Hosting VPS Web Hosting Managed Web Hosting
  Cloud Web Hosting Dedicated Server E-commerce Web Hosting Cheap Web Hosting


Premium Partners:


Visit forums.thewebhostbiz.com: to discuss the web hosting business, buy and sell websites and domain names, and discuss current web hosting tools and software.