Date Added: Dec 2010
The up-to-date web-world has become more complex in size and information relating to various aspects within its sphere. The human being is now in a cultural habit of searching the web for information. Search engine is also one of the techniques which help the human empirical nature. Crawling is a procedure through which search engine crawls the web, and stores the necessary document and their corresponding URL in the back end. This paper presented here is all about a development of a crawler which is hierarchical in nature even by maintaining parallelism during classification and analysis of the web page.