Abstract

The Air Travel Information System (ATIS) domain serves as the common evaluation task for ARPA"spoken language system developers. 1 To support this task, the Multi-Site ATIS Data COl-lection Working group (MADCOW) coordinates data collection activities. This paper describes recent MADCOW activities. In particular, this paper describes the migration of the ATIS task to a richer relational database and development corpus (ATIS-3) and describes the ATIS-3 corpus. The expanded database, which includes information on 46 US and Canadian cities and 23,457 flights, was released in the fall of 1992, and data collection for the ATIS-3 corpus began shortly thereafter. The ATIS-3 corpus now consists of a total of 8297 released training utterances and 3211 utterances reserved for testing, collected at BBN, CMU, MIT, NIST and SRI. 2906 of the training utterances have been anno-tated with the correct information from the database. This paper describes the ATIS-3 corpus in detail, including breakdowns of data by type (e.g. context-independent, context-dependent, and unevaluable)and variations in the data collected at different sites. This paper also includes a description of the ATIS-3 database. Finally, we discuss future data collection and evaluation plans. 1. BACKGROUND The ATIS task was first used as a common ARPA spoken language evaluation task in 1990 [1,2]. In the ATIS task, subjects obtain air travel information such as flight sched-ules, fares, and ground transportation from a relational database using spoken natural language, and use it to solve air travel planning scenarios. Although the core air travel planning task has remained the same since the beginning, its use in evaluation has gradually evolved over the years with the general objectives of increasing tile match between 1. This paper was prepared under the auspices of the Multi-Site ATIS Data Collection Working group (MADCOW). In addition to the authors, many other people, listed under the Acknowledgments section, made important contributions to this work.

Citations