Abstract

Many different metrics exist for evaluating parsing results, including Viterbi, Crossing Brackets Rate, Zero Crossing Brackets Rate, and several others. However, most parsing algorithms, including the Viterbi algorithm, attempt to optimize the same metric, namely the probability of getting the correct labelled tree. By choosing a parsing algorithm appropriate for the evaluation metric, better performance can be achieved. We present two new algorithms: the "Labelled Recall Algorithm," which maximizes the expected Labelled Recall Rate, and the "Bracketed Recall Algorithm," which maximizes the Bracketed Recall Rate. Experimental results are given, showing that the two new algorithms have improved performance over the Viterbi algorithm on many criteria, especially the ones that they optimize.

Citations

316	The estimation of stochastic context-free grammars using the inside-outside algorithm. Computer Speech and Language - Lari, Young - 1990
240	Inside-Outside Reestimation from Partially Bracketed Corpora - Pereira, Schabes - 1992
232	Trainable Grammars for Speech Recognition - Baker - 1979
158	A Corpus-Based Approach to Language Learning - Brill - 1993
136	Natural Language Parsing as Statistical Pattern Recognition - Magerman - 1994
51	Efficient algorithms for parsing the DOP model - Goodman - 1996
43	Using an annotated corpus as a stochastic grammar - Bod - 1993
43	Efficiency, robustness, and accuracy in picky chart parsing - Magerman, Weir - 1992