The paper presents principles of dialogue co-operativity derived from a corpus of task-oriented spoken human-machine dialogue. The corpus was recorded during the design of a dialogue model for a spoken language dialogue system. Analysis of the corpus produced a set of dialogue design principles intended to prevent users from having to initiate clarification and repair meta-communication which the system would not understand. Developed independently of Grice's work on co-operation in spoken dialogue, these principles provide an empirical test of the correctness and completeness of Grice's maxims of co-operativity in the case of human-machine dialogue. Whereas the maxims pass the test of correctness, they fail to provide a complete account of principles of co-operative human-machine dialogue. A more complete set of aspects of co-operative taskoriented dialogue is proposed together with the principles expressing those aspects. Transferability of results to co-operative spoken human-human ...

cooperative spoken human-machine dialogue which are based on the development and controlled user testing of the dialogue component of the Danish dialogue system as well as on comparison with human-human dialogue theory. Potentially, the principles could be used as effective and systematic dialogue development and evaluation tools both during early design and in later phases of dialogue evaluation.

This paper reports the results of the development, deployment and testing of a large spoken-language dialogue application for use by the general public. We built an automated spoken questionnaire for the U.S. Bureau of the Census. In the project's first phase, the basic recognizers and dialogue system were developed using 4,000 calls. In the second phase, the system was adapted to meet Census Bureau requirements and deployed in the Bureau's 1995 national test of new technologies. In the third phase, we refined the system and showed empirically that an automated spoken questionnaire could successfully collect and recognize census data, and that subjects preferred the spoken system to written questionnaires. Our large data collection effort and two subsequent field tests showed that, when questions are asked correctly, the answers contain information within the desired response categories about 99 percent of the time. 1 Introduction Every ten years, the U.S. Bureau of the Census (hereaf...

Spoken language dialogue systems technologies are beginning to master the design and implementation of applied systems for complex well-structured tasks. Partly for this reason, there is a need for evaluation metrics which include general concepts of task and dialogue types. The paper reports on the scenario-based user test of the dialogue management of an airline ticket reservation system. The test data are compared to the data from the last Wizard of Oz iteration before the system was implemented. Detailed analysis of user dialogue behaviour reveals a series of principled limitations of systemdirected dialogue for complex well-structured tasks. The discussion weighs those limitations against the demonstrated potential of system-directed dialogue for a broad class of tasks. 1.

Controlled user testing of the dialogue component of spoken language dialogue systems (SLDSs) has a natural focus on the detection, analysis and repair of dialogue design problems. Not only dialogue designers and their systems commit errors, however. Users do so as well. Improvement of dialogue interaction is not only a matter of reducing the number and severity of dialogue design problems but also of preventing the occurrence of avoidable user errors. Based on a controlled user test of the dialogue component of an implemented SLDS, the paper takes a systematic look at the dialogue errors made by users in the test corpus. A typology of user errors in spoken human-machine dialogue is presented and discussed, and potentially important dialogue design advice derived from the fact that the notion of a `user error' turns out to be one that must be handled with care.

We present and apply an empirical methodology for evaluating the effectiveness of dialogues in spoken language systems. This methodology is suitable in particular for evaluation of dialogue-based systems that collect information from the user, such as an automated spoken questionnaire. Our method for assessing effectiveness involves coding answers from users for responsiveness. For this effort, we developed a behavioral coding scheme tailored to the requirements of automated spoken questionnaires interacting via the telephone. The codes cover a range of behavior from "Concise" to "No response." We have used this evaluation methodology in the development of an automated spoken questionnaire. In connection with this project, we collected over 4,000 telephone calls responding to the questionnaire. A sample of the calls was transcribed and coded using our behavioral coding scheme. We then used the data from the codes to choose among alternative protocols for the dialogue and to evaluate di...

This paper presents a principled approach to reducing the occurrence of communication failure in spoken language dialogue systems. A set of principles for cooperative human-machine dialogue has been developed based on the development of the dialogue component of a spoken language dialogue system and on human-human dialogue theory. The principles have been tested on the dialogue corpus from a controlled user test of the implemented system. The paper demonstrates how the principles enabled systematic classification and analysis of the user test data on system miscommunication. In addition, the user test confirmed the broad scope of the principles as only minor additions and revisions were needed to provide a complete classification of the test data. The principles may have other uses in addition to that of test data analysis and dialogue evaluation. Potentially, they might serve as guidelines for the design of cooperative dialogue during early dialogue design.

Dialogue model design for spoken language dialogue systems (SLDSs) is still based mainly on common sense, experience and intuition, and trial and error, rather than on established design principles. Co-operativity in dialogue is crucial to habitable human-machine spoken dialogue. The paper presents a set of principles of co-operative user-system dialogue which have been derived from a corpus of task-oriented spoken human-machine dialogue. The set of principles is shown to include as a sub-set an established body of principles of co-operative human-human dialogue. Analysis of results from a user test of an implemented SLDS prototype shows the set of principles to be adequate to account for the dialogue problems identified in the test corpus. Both empirical and theoretical grounds thus indicate that the principles presented in the paper may constitute a comprehensive set of guidelines for the design of co-operative human-machine dialogue. 1.

We report on utterance verification of putative recognitions in both open-set and closed-set recognition tasks using telephone speech. For open-set recognition, we report on rejection of out-of-vocabulary utterances. In a two-keyword task ("male" and "female") using 50% out-ofvocabulary utterances, utterance verification reduced errors by 60%, from 12% to 4.8% compared to our baseline rejection strategy. For closed-set recognition, we report on re-ordering the N-best hypotheses. In a 58-phrase task, utterance verification reduced closed-set recognition errors by 30%, from 6.5% to 4.5%. 1. INTRODUCTION Recognition based on the combination of phonetic likelihoods from short fixed-width frames is the dominate paradigm for speech recognition systems. While this approach has numerous advantages, it is reasonable to think that better word-level recognition is possible using whole-word classifiers. Building such recognizers presents a number of difficulties, such as finding word boundaries ...

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : xiii 1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1.1 Research Goals : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1.2 Male/Female Versus Last Names : : : : : : : : : : : : : : : : : : : : : : : : 2 1.3 Scaling Up: 58 Phrases : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4 1.4 Vocabulary Independence : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5 1.5 Thesis Overview : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6 1.6 Tutorial on Automatic Speech Recognition : : : : : : : : : : : : : : : : : : : 7 1.6.1 A Setting for Automatic Speech Recognition : : : : : : : : : : : : : 7 1.6.2 Overview of Speech Recognition : : : : : : : : : : : : : : : : : : : : 8 1.6.3 Artificial Neural Network : : : : : : : : : : : : : : : : : : : : : : : : 12 1.6.4 Context-Dependent Modeling : : : : : : : : : : : : : ...