We introduce a memory-based approach to part of speech tagging. Memory-based learning is a form of supervised learning based on similarity-based reasoning. The part of speech tag of a word in a particular context is extrapolated from the most similar cases held in memory. Supervised learning approaches are useful when a tagged corpus is available as an example of the desired output of the tagger. Based on such a corpus, the tagger-generator automatically builds a tagger which is able to tag new text the same way, diminishing development time for the construction of a tagger considerably. Memory-based tagging shares this advantage with other statistical or machine learning approaches. Additional advantages specific to a memory-based approach include (i) the relatively small tagged corpus size sufficient for training, (ii) incremental learning, (iii) explanation capabilities, (iv) flexible integration of information in case representations, (v) its non-parametric nature, (vi) reasonably good results on unknown words without morphological analysis, and (vii) fast learning and tagging. In this paper we show that a large-scale application of the memory-based approach is feasible: we obtain a tagging accuracy that is on a par with that of known statistical approaches, ad with attractive space and time complexity properties when using IGTree, a tree-based formalism for indexing and searching huge case bases. The use of IGTree has as additional advantage that optimal context size for disambiguation is dynamically computed.

A data-oriented (empiricist) alternative to the currently pervasive (nativist) Principles and Pa-rameters approach to the acquisition of stress assignment is investigated. A similarity-based algorithm, viz. an augmented version of Instance-Based Learning is used to learn the system of main stress assignment in Dutch. In this nontrivial task a comprehensive lexicon of Dutch monomorphemes is used instead of the idealized and highly simplified description of the empirical data used in previous approaches. It is demonstrated that a similarity-based learning method is effective in learning the complex stress system of Dutch. The task is accomplished without the a priori knowledge assumed to pre-exist in the learner in a Principles and Parameters framework. A comparison of the system's behavior with a consensus linguistic analysis (in the framework of Metrical Phonology) shows that ease of learning correlates with decreasing degrees of marked-ness of metrical phenomena. It is also shown that the learning algorithm captures subregularities within the stress system of Dutch that cannot be described without going beyond some of the theoretical assumptions of metrical phonology.

Current approaches to computational lexicology in language technology are knowledge-based (competence-oriented) and try to abstract away from specific formalisms, domains, and applications. This results in severe complexity, acquisition and reusability bottlenecks. As an alternative, we propose a particular performance-oriented approach to Natural Language Processing based on automatic memory-based learning of linguistic (lexical) tasks. The consequences of the approach for computational lexicology are discussed, and the application of the approach on a number of lexical acquisition and disambiguation tasks in phonology, morphology and syntax is described.

It is traditionally assumed that various sources of linguistic knowledge and their interaction should be formalised in order to be able to convert words into their phonemic representations with reasonable accuracy. We show that using supervised learning techniques, based on a corpus of transcribed words, the same and even better performance can be achieved, without explicit modeling of linguistic knowledge. In this paper we present two instances of this approach. A first model implements a variant of instance-based learning, in which a weighed similarity metric and a database of prototypical exemplars are used to predict new mappings. In the second model, graphemeto -phoneme mappings are looked up in a compressed text-to-speech lexicon (table lookup) enriched with default mappings. We compare performance and accuracy of these approaches to a connectionist (backpropagation) approach and to the linguistic knowledge-based approach.

In this paper we discuss the application of Memory-Based Learning (MBL) to fast NP chunking. We first discuss the application of a fast decision tree variant of MBL (IGTree) on the dataset described in (Ramshaw and Marcus, 1995), which consists of roughly 50,000 test and 200,000 train items. In a second series of experiments we used an architecture of two cascaded IGTrees. In the second level of this cascaded classifier we added context predictions as extra features so that incorrect predictions from the first level can be corrected, yielding a 97.2% generalisation accuracy with training and testing times in the order of seconds to minutes. Submission Type: regular paper Topic Areas: robust parsing, NP chunking, memory-based learning Author of Record: Jorn Veenstra Under consideration for other conferences (specify)? no Fast NP Chunking Using Memory-Based Learning Techniques Abstract In this paper we discuss the application of Memory-Based Learning (MBL) to fast NP chunking. We fir...

This paper investigates the computational grounding of learning theories developed within a metrical phonology approach to stress assignment. In current research, the Principles and Parameters approach to learning stress is pervasive. We point out some inherent problems associated with this approach in learning the stress system of a particular language by setting parameters (the case of Dutch), which is shown to be an inherently noisy problem. The paper focuses on two aspects of this problem: we empirically examine the effect of input encodings on learnability, and we investigate the possibility of a data-oriented approach as an alternative to the principles and parameters approach. We show that data-oriented similarity-based machine learning techniques like Backpropagation Learning, Instance-Based Learning and Analogical Modeling working on phonemic input encodings (i) are able to learn metrical phonology abstractions based on concepts like syllable weight, (ii) that their performance can be related to various degrees of markedness of metrical phenomena, and (iii) that in addition, they are able to extract generalizations which cannot be expressed within the metrical framework without recourse to lexical marking. We also provide a quantitative comparison of the performance of the three algorithms investigated.

We provide a qualitative and empirical comparison of Skousen's Analogical Modeling algorithm (AM) with Lazy Learning (LL) on a typical Natural Language Processing task. AM incorporates an original approach to feature selection and to the handling of symbolic, unordered feature values. More specifically, it provides a method to dynamically compute an optimally-sized set of nearest neighbours (the analogical set) for each test item, on the basis of which the most plausible category can be selected. We investigate the algorithm's generalisation accuracy and its tolerance to noise and compare it to Lazy Learning techniques on a primary stress assignment task in Dutch. The latter problem is typical for a large amount of classification problems in Natural Language Processing. It is shown that AM is highly successful in performing the task: it outperforms Lazy Learning in its basic scheme. However, LL can be augmented so that it performs at least as well as AM and becomes as noise tolerant as...

Abstract not found

In this paper, a data-oriented model of text-to-speech processing is described. On the basis of a large text-to-speech corpus, the model automatically gathers a distributed, yet symbolic representation of subword-phoneme association knowledge, representing this knowledge in the form of paths in a decision tree. Paths represent context-sensitive rewrite rules which unambiguously map strings of letters onto single phonemes. The more ambiguous the mapping is, the larger the stored context. The knowledge needed for converting a spelling word to its phonemic transcription is thus represented in a distributed fashion: many different paths contribute to the phonemisation of a word, and a single path may contribute to phonemisations of many words. Some intrinsic properties of the data-oriented model are shown to have relations with psycholinguistic concepts such as a language's orthographic depth, and word pronunciation consistency.

Memory-Based Language Processing (MBLP), like Analogical Modeling of Language (AML), is an approach to modeling language learning and language processing that is based on the idea that language behavior is guided by the direct reuse of memory traces of earlier language experience rather than by rules extracted from such experience. Despite their similarities, both approaches show important theoretical, algorithmic, and empirical differences. MBLP uses algorithms and metrics taken from statistical pattern recognition (nearest neighbor methods), and information theory. AML is based on a natural (psychologically plausible) statistic. We will discuss these differences, focusing on new empirical work comparing AML and MBLP on learning and processing plural formation in German. 1