Pivoted Document Length Normalization (1996) [261 citations — 17 self]
Popular Tags
No tags have been applied to this document.
BibTeX | Add To MetaCart
author = {Amit Singhal and Chris Buckley and Mandar Mitra and Ar Mitra},
title = {Pivoted Document Length Normalization},
booktitle = {},
year = {1996},
pages = {21--29},
publisher = {ACM Press}
}
Years of Citing Articles
Bookmarks
OpenURL
Abstract:
Automatic information retrieval systems have to deal with documents of varying lengths in a text collection. Document length normalization is used to fairly retrieve documents of all lengths. In this study, we observe that a normalization scheme that retrieves documents of all lengths with similar chances as their likelihood of relevance will outperform another scheme which retrieves documents with chances very different from their likelihood of relevance. We show that the retrieval probabilities for a particular normalization method deviate systematically from the relevance probabilities across different collections. We present pivoted normalization, a technique that can be used to modify any normalization function thereby reducing the gap between the relevance and the retrieval probabilities. Training pivoted normalization on one collection, we can successfully use it on other (new) text collections, yielding a robust, collection independent normalization technique. We use the idea o...
