Automated Summarization Evaluation with Basic Elements (2006)
| Venue: | In Proceedings of the Fifth Conference on Language Resources and Evaluation (LREC |
| Citations: | 32 - 3 self |
BibTeX
@INPROCEEDINGS{Hovy06automatedsummarization,
author = {Eduard Hovy and Chin-yew Lin and Liang Zhou and Junichi Fukumoto},
title = {Automated Summarization Evaluation with Basic Elements},
booktitle = {In Proceedings of the Fifth Conference on Language Resources and Evaluation (LREC},
year = {2006}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
As part of evaluating a summary automatically, it is usual to determine how much of the contents of one or more human-produced ‘ideal ’ summaries it contains. Past automated methods such as ROUGE compare using fixed word ngrams, which are not ideal for a variety of reasons. In this paper we describe a framework in which summary evaluation measures can be instantiated and compared, and we implement a specific evaluation method using very small units of content, called Basic Elements, that address some of the shortcomings of ngrams. This method is tested on DUC 2003, 2004, and 2005 systems
Citations
| 992 | BLEU: A Method for Automatic Evaluation of Machine Translation - Papineni, Roukos, et al. - 2002 |
| 260 | and E.Hovy. Automatic Evaluation of Summaries Using N-gram Co-Occurrence Statistics - Lin - 2003 |
| 114 | A dependency-based method for evaluating broad-coverage parsers - Lin - 1995 |
| 91 | Evaluating content selection in summarization: The pyramid method - NENKOVA, R |
| 6 | and automatic evaluation of summaries - Manual |
