Measuring Efficiency in High-Accuracy, Broad-Coverage Statistical Parsing
| Venue: | In Proceedings of the Efficiency in Largescale Parsing Systems Workshop at COLING-2000 |
| Citations: | 3 - 2 self |
BibTeX
@INPROCEEDINGS{Roark_measuringefficiency,
author = {Brian Roark and Eugene Charniak},
title = {Measuring Efficiency in High-Accuracy, Broad-Coverage Statistical Parsing},
booktitle = {In Proceedings of the Efficiency in Largescale Parsing Systems Workshop at COLING-2000},
year = {}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
Very little attention has been paid to the comparison of efficiency between high accuracy statistical parsers. This paper proposes one machine-independent metric that is general enough to allow comparisons across very different parsing architectures. This metric, which we call "events considered", measures the number of "events", however they are defined for a particular parser, for which a probability must be calculated, in order to find the parse. It is applicable to single-pass or multi-stage parsers. We discuss the advantages of the metric, and demonstrate its usefulness by using it to compare two parsers which differ in several fundamental ways. 1 Introduction The past five years have seen enormous improvements in broad-coverage parsing accuracy, through the use of statistical techniques. The parsers that perform at the highest level of accuracy (Charniak (1997; 2000); Collins (1997; 2000); Ratnaparkhi, 1997) use probabilistic models with a very large number of parameters, which...
