A Study of Mobile Agent Platforms for Distributed Web Crawling

Provided by: International Journal of Advances in Engineering Science & Technology
Topic: Big Data
Format: PDF
In the traditional centralized crawling techniques the pages from all over the web are brought to the search engine side which results a lot of unnecessary Internet traffic. In the distributed crawling with migrating agents, the mobile agent is send to the web server side that brings only required pages to the search engine side which reduces unnecessary overhead. The mobile (migrating) agents are processes capable of roaming a wide area networks, interacting with foreign hosts, gathering information on behalf of its owner and coming back having performed the duties set by its user.

Find By Topic