Evaluating DUC 2005 using Basic Elements (2005)
by
Eduard Hovy
,
Chin-yew Lin
,
Liang Zhou
| Venue: | Proceedings of DUC-2005 |
| Citations: | 27 - 3 self |
BibTeX
@INPROCEEDINGS{Hovy05evaluatingduc,
author = {Eduard Hovy and Chin-yew Lin and Liang Zhou},
title = {Evaluating DUC 2005 using Basic Elements},
booktitle = {Proceedings of DUC-2005},
year = {2005}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
In this paper we introduce Basic Elements, a new way of automating the evaluation of text summaries. We show that this method correlates better with human judgments than any other automated procedure to date, and overcomes the subjectivity/variability problems of manual methods that require humans to preprocess summaries to be evaluated. This is demonstrated on DUC 2005 peer systems and
Citations
| 992 | BLEU: A Method for Automatic Evaluation of Machine Translation - Papineni, Roukos, et al. - 2002 |
| 780 | Head-Driven Statistical Models for Natural Language Parsing - Collins - 1999 |
| 671 | A Maximum-Entropy-Inspired Parser - Charniak - 2000 |
| 260 | and E.Hovy. Automatic Evaluation of Summaries Using N-gram Co-Occurrence Statistics - Lin - 2003 |
| 136 | Natural Language Parsing as Statistical Pattern Recognition - Magerman - 1994 |
| 114 | A dependency-based method for evaluating broad-coverage parsers - Lin - 1995 |
| 91 | Evaluating content selection in summarization: The pyramid method - NENKOVA, R |
| 56 | Intelligent writing assistance - Heidorn - 2000 |
| 12 | Evaluation method for automatic speech summarization - Hori, Hori, et al. - 2003 |
| 6 | and automatic evaluation of summaries - Manual |
| 3 | Summarization evaluation via relative utility - Radev, Tam - 2003 |
