Overview of the trec 2007 question answering track (2007)
| Citations: | 22 - 1 self |
BibTeX
@MISC{Dang07overviewof,
author = {Hoa Trang Dang and Diane Kelly and Jimmy Lin},
title = {Overview of the trec 2007 question answering track},
year = {2007}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
The TREC 2007 question answering (QA) track contained two tasks: the main task consisting of series of factoid, list, and “Other ” questions organized around a set of targets, and the complex, interactive question answering (ciQA) task. The main task differed from previous years in that the document collection comprised blogs in addition to newswire documents, requiring systems to process diverse genres of unstructured text. The evaluation of factoid and list responses distinguished between answers that were globally correct (with respect to the document collection), and those that were only locally correct (with respect to the supporting document but not to the overall document collection). The ciQA task provided a framework for participants to investigate interaction in the context of complex information needs. Standing in for surrogate users, assessors interacted with QA systems live over the Web; this setup allowed participants to experiment with more complex interfaces but also revealed limitations in the ciQA design for evaluation of interactive systems. 1
Citations
| 30 | Answering definition questions with multiple knowledge sources - Hildebrandt, Katz, et al. - 2004 |
| 29 | Overview of the TREC 2005 Question Answering Track - Voorhees, Dang - 2005 |
| 25 | Overview of the TREC 2006 Question Answering Track - Dang, Lin, et al. - 2006 |
| 17 | track overview in TREC 2005: High Accuracy Retrieval from Documents - HARD - 2005 |
| 11 | pyramids built of nuggets topple over - Will - 2006 |
| 6 | Fumito Masui, and Noriko Kando. 2004. Handling information access dialogue through QA technologies—A novel challenge for open-domain question answering - Kato, Fukumoto |
| 2 | Overview of the TREC 2004 question answering track - 2005a |
| 1 | Using question series to evaluate question answering system effectiveness - 2005b |
