DMCA
Letor: Benchmark dataset for research on learning to rank for information retrieval (2007)
Cached
Download Links
| Venue: | In Proceedings of SIGIR 2007 Workshop on Learning to Rank for Information Retrieval |
| Citations: | 156 - 16 self |
BibTeX
@INPROCEEDINGS{Liu07letor:benchmark,
author = {Tie-yan Liu and Jun Xu and Tao Qin and Wenying Xiong and Hang Li},
title = {Letor: Benchmark dataset for research on learning to rank for information retrieval},
booktitle = {In Proceedings of SIGIR 2007 Workshop on Learning to Rank for Information Retrieval},
year = {2007}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
This paper is concerned with learning to rank for information retrieval (IR). Ranking is the central problem for information retrieval, and employing machine learning techniques to learn the ranking function is viewed as a promising approach to IR. Unfortunately, there was no benchmark dataset that could be used in comparison of existing learning algorithms and in evaluation of newly proposed algorithms, which stood in the way of the related research. To deal with the problem, we have constructed a benchmark dataset referred to as LETOR and distributed it to the research communities. Specifically we have derived the LETOR data from the existing data sets widely used in IR, namely, OHSUMED and TREC data. The two collections contain queries, the contents of the retrieved documents, and human judgments on the relevance of the documents with respect to the queries. We have extracted features from the datasets, including both conventional features, such as term frequency, inverse document frequency, BM25, and language models for IR, and features proposed recently at SIGIR, such as HostRank, feature propagation, and topical PageRank. We have then packaged LETOR with the extracted features, queries, and relevance judgments. We have also provided the results of several state-ofthe-arts learning to rank algorithms on the data. This paper describes in details about LETOR.
Keyphrases
information retrieval benchmark dataset central problem trec data conventional feature promising approach retrieved document related research research community language model extracted feature ranking function data set term frequency several state-ofthe-arts topical pagerank letor data inverse document frequency feature propagation relevance judgment human judgment
