Query-Driven Indexing for Scalable Peer-to-Peer Text Retrieval

DMCA

Query-Driven Indexing for Scalable Peer-to-Peer Text Retrieval (2007)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Gleb Skobeltsyn , Toan Luu , Ivana Podnar ˇ Zarko , Martin Rajman , Karl Aberer

Venue:	In Infoscale
Citations:	9 - 3 self

BibTeX

@INPROCEEDINGS{Skobeltsyn07query-drivenindexing,
    author = {Gleb Skobeltsyn and Toan Luu and Ivana Podnar ˇ Zarko and Martin Rajman and Karl Aberer},
    title = {Query-Driven Indexing for Scalable Peer-to-Peer Text Retrieval},
    booktitle = {In Infoscale},
    year = {2007}
}

OpenURL

Abstract

We present a query-driven algorithm for the distributed indexing of large document collections within structured P2P networks. To cope with bandwidth consumption that has been identified as the major problem for the standard P2P approach with single term indexing, we leverage a distributed index that stores up to top-k document references only for carefully chosen indexing term combinations. In addition, since the number of possible term combinations extracted from a document collection can be very large, we propose to use query statistics to index only such combinations that are indeed frequently requested by the users. Thus, by avoiding the maintenance of superfluous indexing information, we achieve a substantial reduction in bandwidth and storage. A specific activation mechanism is applied to continuously update the indexing information according to changes in the query distribution, resulting in an efficient, constantly evolving query-driven indexing structure. We show that the size of the index and the generated indexing/retrieval traffic remains manageable even for websize document collections at the price of a marginal loss in precision for rare queries. Our theoretical analysis and experimental results provide convincing evidence about the feasibility of the query-driven indexing strategy for large scale P2P text retrieval. Moreover, our experiments confirm that the retrieval performance is only slightly lower than the one obtained with state-of-the-art centralized query engines.

Keyphrases

query-driven indexing scalable peer-to-peer text retrieval document collection marginal loss query-driven algorithm websize document collection indexing information distributed indexing retrieval performance rare query theoretical analysis state-of-the-art centralized query engine single term indexing standard p2p approach p2p network bandwidth consumption large scale p2p text retrieval query-driven indexing structure generated indexing retrieval traffic major problem substantial reduction possible term combination specific activation mechanism distributed index indexing term combination query statistic large document collection query-driven indexing strategy query distribution experimental result top-k document reference

DMCA