The last decade has witnessed the emergence of a new breed of human computer interfaces that combines several human language technologies to enable information access and transactional processing using spoken dialogue. In this paper, I discuss my view on the research issues involved in the development of such interfaces, describe the recent work done in this area at the MIT Laboratory for Computer Science, and outline some of the unmet research challenges, including the need to work in real domains, spoken language generation, and portability across domains and languages.

This article presents a unified approach to Chinese statistical language modeling (SLM). Applying SLM techniques like trigram language models to Chinese is challenging because (1) there is no standard definition of words in Chinese; (2) word boundaries are not marked by spaces; and (3) there is a dearth of training data. Our unified approach automatically and consistently gathers a high-quality training data set from the Web, creates a high-quality lexicon, segments the training data using this lexicon, and compresses the language model, all by using the maximum likelihood principle, which is consistent with trigram model training. We show that each of the methods leads to improvements over standard SLM, and that the combined method yields the best pinyin conversion result reported.

Cluster-based n-gram modeling is a variant of normal word-based n-gram modeling. It attempts to make use of the similarities between words. In this paper, we present an empirical study of clustering techniques for Asian language modeling. Clustering is used to improve the performance (i.e. perplexity) of language models as well as to compress language models. Experimental tests are presented for cluster-based trigram models on a Japanese newspaper corpus, and on a Chinese heterogeneous corpus.

Systems that attempt to understand natural human input make nfistakes, even humans. However, humans avoid misunderstandings by confirming doubtful input. Multimodal systems--those that combine simultaneous input from more than one modality, for example speech and gesture----have historically been designed so that they either request confirmation of speech, their primary modality, or not at all. Instead, we experimented with delaying confirmation until after the speech and gesture were combined into a complete multimodal command. In controlled experiments, subjects achieved more commands per minute at a lower error rate when the system delayed confirmation, than compared to when subjects confirmed only speech. In addition, this style of late confirmation rreets the user's expectation that confirmed commands should be executable.

The growth in the quantity of information and services offered online has been phenomenal. Nevertheless, access mechanisms have remained relatively primitive, requiring users to primarily point and click their way through a forest of Web links and to expend valuable cognitive capacities to track the geography of the Web space. Conversational systems can provide an intuitive, flexible multi-modal interface to online resources. The explosive growth of the World Wide Web, the continuing standardization of Web related technologies, and the growing penetration of Internet access enable us to embed a very thin client inside a standard Web browser, making conversational interfaces available to a much wider audience. This paper presents WebGALAXY, a conversational spoken language system for access to selected online resources from within a typical browser. A thin Java based client is employed as the front end with much of the speech and natural language processing occuring on remote servers. 1...

The Yellow Pages Assistant (YPA) is a natural language dialogue system which guides a user through a dialogue in order to retrieve addresses from the Yellow Pages. Part of the work in this project is concerned with the construction of a Backend, i.e. the database extracted from the raw input text that is needed for the online access of the addresses. Here we discuss some aspects involved in this task as well as report on experiences which might be interesting for other projects as well.

Over the past year our group has begun development of telephonebased speech understanding capability for our GALAXY conversational system. An important part of this process has been the collection of telephone speech which was used for training and evaluation. In the first phase of data collection our goal was to collect read speech from a wide variety of talkers, telephone handsets, and noise/channel conditions. In the second phase of data collection our additional goal was to collect spontaneous telephone speech from subjects actually using the system. In order to maximize variation in telephone conditions, as well as ease of use for subjects, the data collection software was designed to telephonesubjects at their specified phonenumbers aroundNorth America. Subjects initiate the data collection session by submitting an electronic form accessible by a WWW browser. For read speech collection, a set of prompts is automatically generated for the subject. This paper describes the design of the data collection systemwe are using for these purposes. To date we have collected over 9,000 utterances from over 270 subjects.

In this paper, we present an approach to lexicon optimization for

We propose a distribution-based pruning of

Cluster-based n-gram modeling is a variant of normal word-based n-gram modeling. It attempts to make use of the similarities between words. In this paper, we present an empirical study of clustering techniques for Asian language modeling. Clustering is used to improve the performance (i.e. perplexity) of language models as well as to compress language models. Experimental tests are presented for cluster-based trigram models on a Japanese newspaper corpus, and on a Chinese heterogeneous corpus. While the majority of previous research on word clustering has focused on how to get the best clusters, we have concentrated our research on the best way to use the clusters. Experimental results show that some novel techniques we present work much better than previous methods, and achieve up to more than 40% size reduction at the same perplexity