Intelligent Crawling on Open Web for Business Prospects
Dynamic nature of web based systems requires continuous system updating. Information retrieval depends upon crawlers that crawl the web exhaustively, but business corporate expect from their crawlers to retrieve the specific information as per their applications. Crawlers help to download the required information using hyperlinks that occur in Web pages but the information is usually partial & fails to fulfill user's aspirations. To retrieve updated information from one single link/URL is very simple but if many URLs give the same information, it becomes difficult to analyze which URL/link is giving desired, sufficient, updated & up to date information.