This report summarizes the work of the Dependency Language Modeling group at the 1996 Summer Speech Workshop at the Center for Language and Speech Processing at Johns Hopkins University (WS96). We motivate and descibe a novel statistical language model that models the syntactic dependencies between words. The model is formulated in the maximum entropy framework, which expresses statistical constraints on the frequencies of various type of dependencies, as well the standard N-gram statistics. We describe how this model was applied to the recognition of spontaneous English speech from the Switchboard corpus. Due to implementation constraints, only a reduced version of our model could be tested so far. The model gave a modest improvement over an N-gram baseline model. A by-product of the project is the Maximim Entropy Modeling Toolkit (MEMT), a freely available software package for domain-independent maximum entropy modeling. 1 Introduction Current state-of-the-art language models for s...

This paper describes an approach for constructing a mixture of language models based on simple statistical notions of semantics using probabilistic models developed for information retrieval. The approach encapsulates corpus-derived semantic information and is able to model varying styles of text. Using such information, the corpus texts are clustered in an unsupervised manner and a mixture of topic-specific language models is automatically created. The principal contribution of this work is to characterise the document space resulting from information retrieval techniques and to demonstrate the approach for mixture language modelling. A comparison is made between manual and automatic clustering in order to elucidate how the global content information is expressed in the space. We also compare (in terms of association with manual clustering and language modelling accuracy) alternative term-weighting schemes and the effect of singular value decomposition dimension reduction (...

A speech recognizer is a device that translates speech into text. Many current speech recognizers contain two components, an acoustic model and a statistical language model. The acoustic model indicates how likely it is that a certain word corresponds to a part of the acoustic signal (e.g. the speech). The statistical language model indicates how likely it is that a certain word will be spoken next, given the words recognized so far. Even though the acoustic model might for example not be able to decide between the acoustically similar words "peach" and "teach", the statistical language model can indicate that the word "peach" is more likely if the previously recognized words are "He ate the". Current speech recognizers perform well on constrained tasks, but the goal of continuous, speaker independent speech recognition in potentially noisy environments with a very large vocabulary has not been reached so far. How can statistical language models be improved so that more complex tasks c...

The prominent modeling technique for speech recognition today is the hidden Markov model with Gaussian emission densities. They have suffered, though, from an inability to learn discriminative information and are prone to overfitting and overparameterization. Recent work on machine learning has moved toward models such as the support vector machine that automatically control generalization and parameterization as part of the overall optimization process. The support vector machine, however, requires ad hoc (and unreliable) methods to couple it to probabilistic speech recognition systems. In this work, we introduce the use of a probabilistic Bayesian learning machine termed the relevance vector machine as the core pattern recognition unit in a speech recognizer. The relevance vector machine system is compared to previous work using support vector machines and is found to outperform the support vector machine system in terms of both accuracy and sparsity on a continuous alphadigit task.

Introduction Grammar-based natural language processing has reached a level where it can `understand ' language to a limited degree in restricted domains. For example, it is possible to parse textual material very accurately and assign semantic relations to parts of sentences. An alternative approach originates from the work of Shannon over half a century ago [41], [42]. This approach assigns probabilities to linguistic events, where mathematical models are used to represent statistical knowledge. Once models are built, we decide which event is more likely than the others according to their probabilities. Although statistical methods currently use a very impoverished representation of speech and language (typically finite state), it is possible to train the underlying models from large amounts of data. Importantly, such statistical approaches often produce useful results. Statistical approaches seem especially well-suited to spoken language which is often spontaneous or conversational

In this paper, we propose adding long-term grammatical information in a Whole Sentence Maximun Entropy Language Model (WSME) in order to improve the performance of the model. The grammatical information was added to the WSME model as features and were obtained from a Stochastic Context-Free grammar. Finally, experiments using a part of the Penn Treebank corpus were carried out and significant improvements were acheived.

14.4> ffl Spoken language: -- audio data. -- often spontaneous (e.g., conversation). -- many expressions that cannot be described by standard grammar. -- may be transcribed by human or speech recogniser. 8th ELSNET summer school 2 Statistical Language Modelling --- Warming Up 24 July 2000 ' & $ % Written Language and Spoken Language (2) `The edge' by Dick Francis : "On Saturday evening and early Sunday morning I packed two bags, the new suitcase from England and a softer holdall bought in Toronto. Into the first I put the rich young owner's suit, cashmere pullover and showy shirts and into the second the new younger-looking clothes for off-duty Tommy, jeans, sweatshirts, woolly hat and trainers. I packed the Scandinavian jersey I'd worn at Woodbine into the suitcase just in case it jogged anyone's memory, and got dressed in dark trousers, open-necked shirt and a s

Abstract not found

Abstract not found

In this paper by utilizing the capabilities of modern ubiquitous operating systems we introduce a comprehensive framework for a ubiquitous translation and language learning environment for English to Sanskrit Machine Translation. We present an application for learning Sanskrit characters, sentences and English Sanskrit translation. For the implementation, we have used the open-source Android platform on the Samsung Mini2440, a state-of-the-art development board. We present our current state of implementation, the architecture of our framework,and the findings we have gathered so far. In addition to this, here we describes the Phrase-Based Statistical Machine Translation Decoder for English to Sanskrit translation in ubiquitous environment. Our goal is to improve the translation quality by enhancing the translation table and by preprocessing the Sanskrit language text. General Terms Artificial intelligence, machine learning, machine translation, statistical machine translation.