PDA

View Full Version : Spiders and Crawlers



Vamp1re
02-13-2013, 05:19 AM
Site crawled by spiders of search engines, Spider is a software of search engine which crawl your site and then save that into the data base of search engine.

easycommerce
02-13-2013, 07:01 AM
If we no need to crawl the spider simply we can add robots.txt file to mention with no follow/no index :)

seoworldclass
02-13-2013, 12:10 PM
Hi Friends,

Spider: A browser that downloads website pages.
Crawler:A Program that follows web pages automatically.

Thanks and Regards
seoworldclass.com

peterwilliams
02-14-2013, 02:31 AM
hey, the detail you shared about the spider and crawl is quite useful. Nowmy doubt about spider is clear.

sabrinasai
02-14-2013, 04:09 AM
A spider, also known as a robot or a crawler, is actually just a program that follows, or "crawls", links throughout the Internet, grabbing content from sites and adding it to search engine indexes.

Spiders only can follow links from one page to another and from one site to another. That is the primary reason why links to your site (inbound links) are so important. Links to your website from other websites will give the search engine spiders more "food" to chew on. The more times they find links to your site, the more times they will stop by and visit. Google especially relies on its spiders to create their vast index of listings.

stephanwally
02-14-2013, 05:31 AM
This is major one thing they divided it into pieces so called it with different names .

lucyrai1
02-14-2013, 05:53 AM
A spider, also known as a robot or a crawler, is actually just a program that follows, or "crawls", links throughout the Internet, grabbing content from sites and adding it to search engine indexes.

Spiders only can follow links from one page to another and from one site to another. That is the primary reason why links to your site (inbound links) are so important. Links to your website from other websites will give the search engine spiders more "food" to chew on. The more times they find links to your site, the more times they will stop by and visit. Google especially relies on its spiders to create their vast index of listings.


I really appreciate this information dear.. Never read about it like this type of post. Awesome dear..

jamsen
02-27-2013, 05:18 PM
A spider is a program that visits Web sites and reads their pages and other ... on the Web all have such a program, which is also known as a "crawler" or a "bot.

tomm121
02-28-2013, 02:21 AM
A spider, also known as a robot or a crawler, is actually just a program that follows, or "crawls", links throughout the Internet, grabbing content from sites and adding it to search engine indexes.

Spiders only can follow links from one page to another and from one site to another. That is the primary reason why links to your site (inbound links) are so important. Links to your website from other websites will give the search engine spiders more "food" to chew on. The more times they find links to your site, the more times they will stop by and visit. Google especially relies on its spiders to create their vast index of listings.



Hello,


Thanks for this reply. I really appreciate this answer.......

Raju
02-28-2013, 02:48 AM
A Web crawler is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing.

raveinfosys
02-28-2013, 03:21 AM
A spider is a program run by a search engine to build a summary of a website’s content (content index). Spiders create a text-based summary of content and an address (URL) for each webpage.

When a person searches, the keyword(s) they enter are compared with the available website content indexes. Due to the large number of webpages indexed, direct text-only-matching is rare, rather search engines use sophisticated logics (algorithms) to rank potential matches. For example, the underlying information hierarchy of a webpage (semantic markup) may be factored into the ranking a webpage is assigned.

A spider will only index the text-based content of your website. Images (such as photos or graphical-text), video and Flash elements require supplementary HTML-text to be ‘seen’ by a spider.