The International Journal of Innovative Research in Computer and Communication Engineering
The internet is an effective media for information sharing and propaganda broadcasting. The extraordinary growth of the Internet has resulted in due attention on web crawling techniques in recent years. In spite of huge leaps in communication, storage and work out power in recent years, hidden URL identifiers always fight back to keep up with web content generation and modification. Also there is no specific pattern matches to identify the proper URLs. Hence, there is a need to focus on attempting to speed up the traversal process, to increase the production of high quality pages and to allocate appropriate tribute to different content along a traversal path.