Web Crawler: Extracting the Web Data

Internet usage has increased a lot in recent times. Users can find their resources by using different hypertext links. This usage of Internet has led to the invention of web crawlers. Web crawlers are full text search engines which assist users in navigating the web. These web crawlers can also be used in further research activities. For e.g. the crawled data can be used to find missing links, community detection in complex networks. In this paper, the authors have reviewed web crawlers: their architecture, types and various challenges being faced when search engines use the web crawlers.

Provided by: IJCTT-International Journal of Computer Trends and Technology Topic: Data Management Date Added: Jul 2014 Format: PDF

Find By Topic