Searching a Terabyte of Text Using Partial Replication

Searching a Terabyte of Text Using Partial Replication (1999)

Cached

Download Links

by Zhihong Lu , Kathryn S. McKinley

Citations:

1 - 1 self

Versions

Version History

Metadata Version 4

User correction supplied by mph

Datum	Value	Source
TITLE	Searching a Terabyte of Text Using Partial Replication	user correction - Legacy Corrections
AUTHOR NAME	Zhihong Lu	user correction
AUTHOR AFFIL	Department of Computer Science, University of Massachusetts, Amherst	user correction
AUTHOR ADDR	MA 01003	user correction
AUTHOR NAME	Kathryn S. McKinley	user correction
AUTHOR AFFIL	Department of Computer Science, University of Massachusetts, Amherst	user correction
AUTHOR ADDR	MA 01003	user correction
ABSTRACT	The explosion of content in distributed information retrieval (IR) systems requires new mechanisms in order to attain timely and accurate retrieval of unstructured text. In this paper, we investigate using partial replication to search a terabyte of text in our distributed IR system. We use a replica selection database to direct queries to relevant replicas that maintain query effectiveness, but at the same time restricts some searches to a small percentage of data to improve performance and scalability, and to reduce network latency. Using a validated simulator, we compare database partitioning to partial replication with load balancing, and find partial replication is much more effective at decreasing query response time, even with fewer resources, and it requires only modest query locality. We also demonstrate the average query response time under 10 seconds for a variety of work loads with partial replication on a terabyte text database. We further investigate query locality with respect to time, replica size, and replica updating costs using real logs from THOMAS and Excite, and discuss the sensitivity of our results to these sample points.	user correction
YEAR	1999	INFERENCE
VENUE TYPE	TECHREPORT	INFERENCE
TECH	Technical Report TR99-50	INFERENCE
CITATIONS	30 found	ParsCit 1.0