Performance analysis of distributed architectures to index one terabyte of text (2004)
| Venue: | Proc. 26th European Conference on IR Research, volume 2997 of Lecture Notes in Computer Science |
| Citations: | 9 - 2 self |
BibTeX
@INPROCEEDINGS{Cacheda04performanceanalysis,
author = {Fidel Cacheda and Vassilis Plachouras and Iadh Ounis},
title = {Performance analysis of distributed architectures to index one terabyte of text},
booktitle = {Proc. 26th European Conference on IR Research, volume 2997 of Lecture Notes in Computer Science},
year = {2004},
pages = {394--408},
publisher = {Springer}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
Abstract. We simulate different architectures of a distributed Information Retrieval system on a very large Web collection, in order to work out the optimal setting for a particular set of resources. We analyse the effectiveness of a distributed, replicated and clustered architecture using a variable number of workstations. A collection of approximately 94 million documents and 1 terabyte of text is used to test the performance of the different architectures. We show that in a purely distributed architecture, the brokers become the bottleneck due to the high number of local answer sets to be sorted. In a replicated system, the network is the bottleneck due to the high number of query servers and the continuous data interchange with the brokers. Finally, we demonstrate that a clustered system will outperform a replicated system if a large number of query servers is used, mainly due to the reduction of the network load. 1
