PCIR: Combining DHTs and Peer Clusters for Efficient Full-Text P2P Indexing

Date Added: Jun 2010
Format: PDF

Distributed Hash Tables (DHTs) are very efficient for querying based on key look-ups. However, building huge term indexes, as required for IR-style keyword search, poses a scalability challenge for plain DHTs. Due to the large sizes of document term vocabularies, peers joining the network cause huge amounts of key inserts and, consequently, a large number of index maintenance messages. Thus, the key to exploiting DHTs for distributed information retrieval is to reduce index maintenance costs. Various approaches in this direction have been pursued, including the use of hybrid infrastructures, or changing the granularity of the inverted index to peer level. The authors show that indexing costs can be significantly reduced further by letting peers form groups in a self-organized fashion.