Results 1  10
of
17
Learning Regular Languages From Simple Positive Examples
, 2000
"... Learning from positive data constitutes an important topic in Grammatical Inference since it is believed that the acquisition of grammar by children only needs syntactically correct (i.e. positive) instances. However, classical learning models provide no way to avoid the problem of overgeneralizati ..."
Abstract

Cited by 29 (0 self)
 Add to MetaCart
(Show Context)
Learning from positive data constitutes an important topic in Grammatical Inference since it is believed that the acquisition of grammar by children only needs syntactically correct (i.e. positive) instances. However, classical learning models provide no way to avoid the problem of overgeneralization. In order to overcome this problem, we use here a learning model from simple examples, where the notion of simplicity is defined with the help of Kolmogorov complexity. We show that a general and natural heuristic which allows learning from simple positive examples can be developed in this model. Our main result is that the class of regular languages is probably exactly learnable from simple positive examples.
Probabilistic FiniteState Machines  Part I
"... Probabilistic finitestate machines are used today in a variety of areas in pattern recognition, or in fields to which pattern recognition is linked: computational linguistics, machine learning, time series analysis, circuit testing, computational biology, speech recognition and machine translatio ..."
Abstract

Cited by 27 (1 self)
 Add to MetaCart
Probabilistic finitestate machines are used today in a variety of areas in pattern recognition, or in fields to which pattern recognition is linked: computational linguistics, machine learning, time series analysis, circuit testing, computational biology, speech recognition and machine translation are some of them. In part I of this paper we survey these generative objects and study their definitions and properties. In part II, we will study the relation of probabilistic finitestate automata with other well known devices that generate strings as hidden Markov models and ngrams, and provide theorems, algorithms and properties that represent a current state of the art of these objects.
Learning DFA from Simple Examples
, 1997
"... Efficient learning of DFA is a challenging research problem in grammatical inference. It is known that both exact and approximate (in the PAC sense) identifiability of DFA is hard. Pitt, in his seminal paper posed the following open research problem: "Are DFAPACidentifiable if examples are d ..."
Abstract

Cited by 22 (5 self)
 Add to MetaCart
Efficient learning of DFA is a challenging research problem in grammatical inference. It is known that both exact and approximate (in the PAC sense) identifiability of DFA is hard. Pitt, in his seminal paper posed the following open research problem: "Are DFAPACidentifiable if examples are drawn from the uniform distribution, or some other known simple distribution?" [25]. We demonstrate that the class of simple DFA (i.e., DFA whose canonical representations have logarithmic Kolmogorov complexity) is efficiently PAC learnable under the Solomonoff Levin universal distribution. We prove that if the examples are sampled at random according to the universal distribution by a teacher that is knowledgeable about the target concept, the entire class of DFA is efficiently PAC learnable under the universal distribution. Thus, we show that DFA are efficiently learnable under the PACS model [6]. Further, we prove that any concept that is learnable under Gold's model for learning from characteristic samples, Goldman and Mathias' polynomial teachability model, and the model for learning from example based queries is also learnable under the PACS model.
Probabilistic FiniteState Machines  Part II
"... Probabilistic finitestate machines are used today in a variety of areas in pattern recognition, or in fields to which pattern recognition is linked. In part I of this paper, we surveyed these objects and studied their properties. In this part II, we study the relations between probabilistic finit ..."
Abstract

Cited by 12 (2 self)
 Add to MetaCart
Probabilistic finitestate machines are used today in a variety of areas in pattern recognition, or in fields to which pattern recognition is linked. In part I of this paper, we surveyed these objects and studied their properties. In this part II, we study the relations between probabilistic finitestate automata and other well known devices that generate strings like hidden Markov models and n grams, and provide theorems, algorithms and properties that represent a current state of the art of these objects.
PAC Learning under Helpful Distributions
 IN &QUOT;PROC. 8TH INTERNATIONAL WORKSHOP ON ALGORITHMIC LEARNING THEORY  ALT'97,&QUOT; (M. LI AND A. MARUOKA, EDS.), LECTURE NOTES IN ARTIFICIAL INTELLIGENCE 1316
, 1997
"... A PAC model under helpful distributions is introduced. A teacher associates a teaching set with each target concept and we only consider distributions such that each example in the teaching set has a nonzero weight. The performance of a learning algorithm depends on the probabilities of the example ..."
Abstract

Cited by 12 (2 self)
 Add to MetaCart
A PAC model under helpful distributions is introduced. A teacher associates a teaching set with each target concept and we only consider distributions such that each example in the teaching set has a nonzero weight. The performance of a learning algorithm depends on the probabilities of the examples in this teaching set. In this model, an Occam's razor theorem and its converse are proved. The class of decision lists is proved PAC learnable under helpful distributions. A PAC learning model with simple teacher (simplicity is based on programsize complexity) is also defined and the model is compared with other models of teaching.
Learning Meaning Before Syntax
, 2008
"... We present a simple computational model that takes into account semantics for language learning, as motivated by readings in the literature of children’s language acquisition and by a desire to incorporate a robust notion of semantics in the field of Grammatical Inference. We argue that not only is ..."
Abstract

Cited by 7 (3 self)
 Add to MetaCart
(Show Context)
We present a simple computational model that takes into account semantics for language learning, as motivated by readings in the literature of children’s language acquisition and by a desire to incorporate a robust notion of semantics in the field of Grammatical Inference. We argue that not only is it more natural to take into account semantics, but also that semantic information can make learning easier, and can give us a better understanding of the relation between positive data and corrections. We propose a model of meaning and denotation using finitestate transducers, motivated by an example domain of geometric shapes and their properties and relations. We give an algorithm to learn a meaning function and prove that it finitely converges to a correct result under a specific set of assumptions about the transducer and examples. We present and analyze the results of empirical tests of our algorithm with natural language samples in the example domain.
On the Relationship Between Lexical Semantics and Syntax for the Inference of ContextFree Grammars
 Proceedings of the 19th National Conference on Artificial Intelligence ({AAAI
, 2004
"... Contextfree grammars cannot be identified in the limit from positive examples (Gold 1967), yet natural language grammars are more powerful than contextfree grammars and humans learn them with remarkable ease from positive examples (Marcus 1993). Identifiability results for formal languages ign ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
Contextfree grammars cannot be identified in the limit from positive examples (Gold 1967), yet natural language grammars are more powerful than contextfree grammars and humans learn them with remarkable ease from positive examples (Marcus 1993). Identifiability results for formal languages ignore a potentially powerful source of information available to learners of natural languages, namely, meanings. This paper explores the learnability of syntax (i.e. contextfree grammars) given positive examples and knowledge of lexical semantics, and the learnability of lexical semantics given knowledge of syntax. The longterm goal is to develop an approach to learning both syntax and semantics that bootstraps itself, using limited knowledge about syntax to infer additional knowledge about semantics, and limited knowledge about semantics to infer additional knowledge about syntax.
On the relationship between Models for Learning in Helpful Environments
 in Proceedings of ICGI 2000, LNAI 1891
, 2000
"... Abstract. The PAC and other equivalent learning models are widely accepted models for polynomial learnability of concept classes. However, negative results abound in the PAC learning framework (concept classes such as deterministic finite state automata (DFA) are not efficiently learnable in the PAC ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Abstract. The PAC and other equivalent learning models are widely accepted models for polynomial learnability of concept classes. However, negative results abound in the PAC learning framework (concept classes such as deterministic finite state automata (DFA) are not efficiently learnable in the PAC model). The PAC model’s requirement of learnability under all conceivable distributions could be considered too stringent a restriction for practical applications. Several models for learning in more helpful environments have been proposed in the literature including: learning from example based queries [2], online learning allowing a bounded number of mistakes [14], learning with the help of teaching sets [7], learning from characteristic sets [5], and learning from simple examples [12,4]. Several concept classes that are not learnable in the standard PAC model have been shown to be learnable in these models. In this paper we identify the relationships between these different learning models. We also address the issue of unnatural collusion between the teacher and the learner that can potentially trivialize the task of learning in helpful environments. Keywords: Models of learning, Query learning, Mistake bounded learning, PAC learning, teaching sets, characteristic samples, DFA learning.
Leveraging Lexical Semantics to Infer ContextFree Grammars
 IN 7TH EUROPEAN CONFERENCE ON PRINCIPLES AND PRACTICE OF KNOWLEDGE DISCOVERY IN DATABASES (PKDD
, 2003
"... Contextfree grammars cannot be identified in the limit from positive examples (Gold, 1967), yet natural language grammars are more powerful than contextfree grammars and humans learn them with remarkable ease from positive examples (Marcus, 1993). Identifiability results for formal languages i ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
Contextfree grammars cannot be identified in the limit from positive examples (Gold, 1967), yet natural language grammars are more powerful than contextfree grammars and humans learn them with remarkable ease from positive examples (Marcus, 1993). Identifiability results for formal languages ignore a potentially powerful source of information available to learners of natural languages, namely, meanings. This paper explores the learnability of contextfree grammars given positive examples and lexical semantics. That is, the learner has a representation of the meaning of each lexical item.