The last decade has witnessed the emergence of a new breed of human computer interfaces that combines several human language technologies to enable information access and transactional processing using spoken dialogue. In this paper, I discuss my view on the research issues involved in the development of such interfaces, describe the recent work done in this area at the MIT Laboratory for Computer Science, and outline some of the unmet research challenges, including the need to work in real domains, spoken language generation, and portability across domains and languages.

Statistical methods have long been the dominant approach in speech recognition and probabilistic modelling in ASR is now a mature technology. The use of statistical methods in other areas of spoken dialogue is however more recent and rather less mature. This paper reviews spoken dialogue systems from a statistical modelling perspective. The complete system is first presented as a partially observable Markov decision process. The various sub-components are then exposed by introducing appropriate intermediate variables. Samples of existing work are reviewed within this framework, including dialogue control and optimisation, semantic interpretation, goal detection, natural language generation and synthesis.

In our effort to build a speech--to--speech translation system for spontaneous spoken dialogs we have developed several methods to improve the language models of the speech decoder of the system. We attempt to take advantage of natural equivalence word classes, frequently occuring word phrases, and discourse structure. Each of these methods was tested on spontaneous English, German and Spanish human--human dialogs. 1. INTRODUCTION The goal of the JANUS project is multi-lingual machine translation of spontaneously spoken dialogs in a limited domain: two people scheduling a meeting with each other. We are currently working with English, German, and Spanish as source languages and German, English, and Japanese as target languages. Table 1 shows the size of training and test set for the English, German and Spanish Spontaneous Scheduling Task databases (ESST, GSST, SSST) used for all experiments reported in this paper, and the coverage of the dictionary over the test set. 1 ESST GSST SSST...

This article is an introduction to some of the emerging research in the application of corpusbased learning techniques to problems in semantic interpretation. In particular, we focus on two important problems in semantic interpretation, namely, word-sense disambiguation and semantic parsing

This paper presents a probabilistic framework for modelling spoken dialogue systems. On the assumption that the overall system behaviour can be represented as a Markov Decision Process, the optimisation of dialogue management strategy using reinforcement learning is reviewed. Examples of learning behaviour are presented for both dynamic programming and sampling methods, but the latter is preferred. The paper concludes by noting the importance of user simulation models for the practical application of these techniques and the need for developing methods of mapping system features in order to achieve suciently compact state spaces.

In this paper we report on our e orts to combine speech and language processing toward multi-lingual spontaneous speech translation. The ongoing work extends our JANUS system e ort toward handling spontaneous spoken discourse and multiple languages. A major objective of this project is to maximize the number of modules, methods and data structures that are language-independent and extensible to other domains. After an overview of the task, databases and the system architecture we will focus on how speech decoding and natural language processing modules will be integrated in a large-scale multi-lingual speech-to-speech translation system for spontaneous spoken discourse. 1.

d my ignorance in the field. He was always patient, and took the time to explain his answers at a level I could understand. iv Dr. Todd Ward, a colleague of mine at IBM, has also "been there" for me. I cannot count the number of times that Todd helped me figure out a solution to a problem, either mathematical or programming. Whenever I was not sure about a solution to a problem, Todd was my sounding board. I'm sure that his individual research efforts were slowed by our meetings, but that never stopped him from helping me. Todd also acted as a counselor, providing insight on how to complete a doctorate! Former IBMer, Dr. Stephen Della Pietra, is without a doubt the brightest mathematician with whom I have ever worked. Like Salim and Todd, he knows statistical modeling at a much greater depth than I do, and he never minded "bringing down" the level of his explanations to one where I could understand and absorb the material. Stephen was my mentor, and without his expert tutelag

this technical report is to explore the extent to which the paradigm used for speech recognition can be extended to cover the design and implementation of complete spoken dialogue systems

We present in this paper a combination of Machine Learning based Information Retrieval (IR) techniques and stochastic language modelling in a hierarchical system that extracts surface information from text. At the lowest level of this hierarchy, documents and paragraphs are successively routed with IR techniques. At the top level, a stochastic language model extracts the most relevant phrases, and labels the type of information they contain. The approach and preliminary results are demonstrated on a subset of the MUC-6 Scenario Templates task.

We present in this paper a combination of Machine Learning based Information Retrieval (IR) techniques and stochastic language modelling in a hierarchical system that extracts surface information from text. At the lowest level of this hierarchy, documents and paragraphs are successively routed with IR techniques. At the top level, a stochastic language model extracts the most relevant phrases, and labels the type of information they contain. The approach and preliminary results are demonstrated on a subset of the MUC-6 Scenario Templates task. 1 Introduction The extraction of information in textual data, today, relies mainly on linguistic analysis. Successful systems have been demonstrated for many information extraction problems [7]. These systems are usually complex and expensive to build, aimed at limited domains of knowledge and difficult to extend. In order to overcome these problems Information Extraction (IE) systems recently began to integrate machine learning (ML) techniques ...