Multi-Site Data Collection and Evaluation in Spoken Language Understanding (1993)
| Venue: | In Proceedings of the Human Language Technology Workshop |
| Citations: | 17 - 3 self |
BibTeX
@INPROCEEDINGS{Hirschman93multi-sitedata,
author = {L. Hirschman and M. Bates and D. Dahl and W. Fisher and J. Garofolo and D. Pallett and K. Hunicke-smith and P. Price and A. Rudnicky and E. Tzoukermann},
title = {Multi-Site Data Collection and Evaluation in Spoken Language Understanding},
booktitle = {In Proceedings of the Human Language Technology Workshop},
year = {1993},
pages = {pages},
publisher = {Morgan Kaufmann}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
The Air Travel Information System (ATIS) domain serves as the common task for DARPA spoken language system research and development. The approaches and results possible in this rapidly growing area are structured by available corpora, annotations of that data, and evaluation methods. Coordination of this crucial infrastructure is the charter of the Multi-Site ATIS Data COllection Working group (MADCOW) . We focus here on selection of training and test data, evaluation of language understanding, and the continuing search for evaluation methods that will correlate well with expected performance of the technology in applications. 1. Introduction Data availability and evaluation procedures structure research possibilities: the type and amount of training data affects the performance of existing algorithms and limits the development of new algorithms; and evaluation procedures document progress, and force research choices in a world of limited resources. The recent rapid progress in spoke...
Citations
| 184 | A Procedure for Quantitatively Comparing the Syntactic Coverage of English Grammars - Black - 1991 |
| 83 | Evaluation of Spoken Language Systems: the ATIS Domain - Price - 1990 |
| 24 | Experiments in evaluating interactive spoken language systems - Polifroni, Hirschman, et al. - 1992 |
| 15 | Multi-site data collection for a spoken language corpus - Hirschman, Bates, et al. - 1992 |
| 9 | Developing an Evaluation Methodology for Spoken Language Systems - Bates, Boisen, et al. - 1990 |
| 1 | The Cost of Errors in a Spoken Language Systems," submitted to Eurospeech93 - Hirschman, Pao - 1993 |
| 1 | Benchmark Tests for the Spoken Language Program - Pallett, Fiscus, et al. - 1993 |
