Abstract

Latent Semantic Indexing (LSI) uses the Singular Value Decomposition to reduce noisy dimensions and improve the performance of text retrieval systems. Preliminary results have shown modest improvements in retrieval accuracy and recall, but these have mainly explored small collections. In this paper we investigate text retrieval on a large document collections (TREC) and focus on distribution of word norm (magnitude). Our results indicate inadequacy of word representations in LSI space on large collections. We emphasize the query expansion interpretation of LSI and propose a LSI term normalization that achieves better performance on larger collections (TREC and NPL).

Citations