An Approach to Reduce Web Crawler Traffic Using Asp.Net

Provided by: International Journal of Soft Computing and Engineering (IJSCE)
Topic: Software
Format: PDF
Now days search engine transfers the web data from one place to another. They work on client server architecture where the central server manages all the information. A web crawler is a program that extracts the information over the web and sends it to the search engine for further processing. It is found that maximum traffic (approximately 40.1%) is due to the web crawler. The proposed scheme shows how web crawler can reduce the traffic using Dynamic web page and HTTP GET request using asp.net.

Find By Topic