Crawler Also known as Spider or Robot, a crawler i
Post# of 261
Posted On: 12/08/2012 9:04:55 PM
Crawler Also known as Spider or Robot, a crawler is a search engine program that "crawls" the web, collecting data, following links, making copies of fresh and updated sites, and storing URLs in the search engine's Index. This allows search engines to provide faster and more up-to-date listings.
(0)
(0)Scroll down for more posts ▼