SEQUITUR is an algorithm that infers a hierarchical structure from a sequence of discrete symbols by replacing repeated phrases with a grammatical rule that generates the phrase, and continuing this process recursively. The result is a hierarchical representation of the original sequence, which offers insights into its lexical structure. The algorithm is driven by two constraints that reduce the size of the grammar, and produce structure as a by-product. SEQUITUR breaks new ground by operating incrementally. Moreover, the method’s simple structure permits a proof that it operates in space and time that is linear in the size of the input. Our implementation can process 50,000 symbols per second and has been applied to an extensive range of real world sequences. 1.

Many theorists have dismissed a priori the idea that distributional information could play a significant role in syntactic category acquisition. We demonstrate empirically that such information provides a powerful cue to syntactic category membership, which can be exploited by a variety of simple, psychologically plausible mechanisms. We present a range of results using a large corpus of child-directed speech and explore their psychological implications. While our results show that a considerable amount of information concerning the syntac-tic categories can be obtained from distributional information alone, we stress that many other sources of information may also be potential contributors to the identification of syntactic classes. I.

It is perhaps misleading to use the word theory to describe the view of language acquisition and cognitive development, which is the subject of this chapter. This word is used as a matter of convenience; it applies here to what is best characterized as a partially completed program of research—a jigsaw puzzle in which certain pieces have been positioned with

Two gating studies, a forced-choice identification study and 2 series of cross-modal repetition priming experiments, traced the time course of recognition of words with onset embeddings (captain) and short words in contexts that match (cap tucked) or mismatch (cap looking) with longer words. Results suggest that acoustic differences in embedded syllables assist the perceptual system in discriminating short words from the start of longer words. The ambiguity created by embedded words is therefore not as severe as predicted by models of spoken word recognition based on phonemic representations. These additional acoustic cues combine with post-offset information in identifying onset-embedded words in connected speech. An important problem in the perception of connected speech is segmentation: how listeners divide the speech stream into individual lexical units or words. Words in fluent speech are not separated by silence in the same way that printed words are divided by blank spaces, yet connected speech is perceived as a sequence of individual words. This perceptual experience clearly reflects acquired language-specific knowledge, because listeners do not have the

This paper presents an unsupervised approach to lexical acquisition with the goodness measure description length gain (DLG) formulated following classic information theory within the minimum description length (MDL) paradigm. The learning algorithm seeks for an optimal segmentation of an utterance that maximises the description length gain from the individual segments. The resultant segments show a nice correspondence to lexical items (in particular, words) in a natural language like English. Learning experiments on large-scMe corpora (e.g., the Brown corpus) have shown the effectiveness of both the learning algorithm and the goodness measure that guides that learning.

An earlier article [25] discusses the proposition that the storage and processing of information in computers and in brains may often be understood as information compression. A subsequent article [15] criticises the computing aspects of [25] and research on the more specific conjecture that all forms of computing and formal reasoning may usefully be understood as information compression. The present article, which is intended to be intelligible without recourse to earlier articles, answers the main points in [15], tries to correct the many inaccuracies and misconceptions in that article, and discusses related issues. Topics which are discussed include: the way theories are or should be developed; the role of evidence in motivating research; apparent shortcomings in the Turing machine concept as a reason for seeking new principles of computing; the apparent conflict between the idea of `computing as compression' and the fact that computers may create redundancy - and how the contradict...

Statistical data compression requires a stochastic language model which must rapidly adapt to new data as it is encountered. A grammatical inference engine is introduced which satisfies this requirement; it is able to discover structure in arbitrary data using nothing more than the predictions of a simple trigram model. We show that compression may be used as an alternative to perplexity for language model evaluation, and that the information processing techniques employed by our system may reflect what happens in the human brain. 1 Introduction Grammatical inference is the process of programming a computer to automatically infer a grammar for a language [8]. We consider a grammar to be nothing more than a model for some data. Applications such as speech recognition and data compression require a stochastic language model, and well-defined performance measures exist for such models. It is easy to get caught in the trap of building complicated models which utilise various ad hoc techni...

In general a grammar describes a (possibly infinite) set of sentences with a finite structural description. Computational Grammar Induction (CGI) deals with the creation of computational models for identification of these infinite sets on the basis of a finite set of examples. CGI is a field in its own right, with its own internal research

Abstract. This paper describes a novel approach to grammar induction that has been developed within a framework designed to integrate learning with other aspects of computing, AI, mathematics and logic. This framework, called information compression by multiple alignment, unification and search (ICMAUS), is founded on principles of Minimum Length Encoding pioneered by Solomonoff and others. Most of the paper describes SP70, a computer model of the ICMAUS framework that incorporates processes for unsupervised learning of grammars. An example is presented to show how the model can infer a plausible grammar from appropriate input. Limitations of the current model and how they may be overcome are briefly discussed. 1

We have designed and are implementing a system called ReV (Real estate Visualiser) for exploring real estate property listings on the world-wide web. Given the large number of different websites providing listings, each with its own presentation format, and the high-dimensionality of the property space itself, it is difficult to obtain a comprehensive single view of property data on the web. ReV addresses this problem by using grammar induction techniques to automatically learn to parse pages from new websites and collate all of their listings together. It them visualizes this listing data using a map-based colorcoding technique. This work draws together a number of strands from the fields of information visualization, machine learning, and database integration. We also hypothesize that ReV will be adaptable to inducing structures from other types of web data. 1. Introduction The rapid growth in data of all kinds stored in computerized form is bringing a surge of interest in visualiza...