Evaluating Message Understanding Systems: An Analysis of . . .

Evaluating Message Understanding Systems: An Analysis of . . . (1993)

by Nancy Chinchor , Lynette Hirschman , David D. Lewis

Venue:	COMPUTATIONAL LINGUISTICS
Citations:	48 - 2 self

User correction supplied by mph

Datum	Value	Source
TITLE	Evaluating Message Understanding Systems: An Analysis of . . .	user correction
AUTHOR NAME	Nancy Chinchor	user correction
AUTHOR AFFIL	Science Applications International Corp.	user correction
AUTHOR NAME	Lynette Hirschman	user correction
AUTHOR AFFIL	Massachusetts Institute of Technology	user correction
AUTHOR NAME	David D. Lewis	user correction
AUTHOR AFFIL	University of Chicago	user correction
ABSTRACT	This paper describes and analyzes the results of the Third Message Understanding Conference (MUC-3). It reviews the purpose, history, and methodology of the conference, summarizes the participating systems, discusses issues of measuring system effectiveness, describes the linguistic phenomena tests, and provides a critical look at the evaluation in terms of the lessons learned. One of the common problems with evaluations is that the statistical significance of the results is unknown. In the discussion of system performance, the statistical significance of the evaluation results is reported and the use of approximate randomization to calculate the statistical significance of the results of MUC-3 is described	user correction - Legacy Corrections
YEAR	1993	INFERENCE
VENUE	COMPUTATIONAL LINGUISTICS	user correction
VENUE TYPE	JOURNAL	INFERENCE
PAGES	409--449	INFERENCE
VOLUME	19	INFERENCE
CITATIONS	25 found	ParsCit 1.0