Extracting Relevant Snippets From Web Documents Through Language Model Based Text Segmentation

Extracting a query-oriented snippet (or passage) and highlighting the relevant information in long document can help reduce the result navigation cost of end users. While the traditional approach of highlighting matching keywords helps when the search is keyword oriented, finding appropriate snippets to represent matches to more complex queries requires novel techniques that can help characterize the relevance of various parts of a document to the given query, succinctly. In this paper, the authors present a language model based method for accurately detecting the most relevant passages of a given document.