Results 1  10
of
42
Learning indexed families of recursive languages from positive data: a survey
, 2008
"... In the past 40 years, research on inductive inference has developed along different lines, e.g., in the formalizations used, and in the classes of target concepts considered. One common root of many of these formalizations is Gold’s model of identification in the limit. This model has been studied f ..."
Abstract

Cited by 18 (6 self)
 Add to MetaCart
In the past 40 years, research on inductive inference has developed along different lines, e.g., in the formalizations used, and in the classes of target concepts considered. One common root of many of these formalizations is Gold’s model of identification in the limit. This model has been studied for learning recursive functions, recursively enumerable languages, and recursive languages, reflecting different aspects of machine learning, artificial intelligence, complexity theory, and recursion theory. One line of research focuses on indexed families of recursive languages — classes of recursive languages described in a representation scheme for which the question of membership for any string in any of the given languages is effectively decidable with a uniform procedure. Such language classes are of interest because of their naturalness. The survey at hand picks out important studies on learning indexed families (including basic as well as recent research), summarizes and illustrates the corresponding results, and points out links to related fields such as grammatical inference, machine learning, and artificial intelligence in general.
String Extension Learning
, 2009
"... This paper defines a collection of functions which define classes of languages, which have the property that they are identifiable in the limit from positive data from a very simple kind of learner. Furthermore these learners are always incremental, maximally consistent, and locally conservative. Th ..."
Abstract

Cited by 11 (5 self)
 Add to MetaCart
(Show Context)
This paper defines a collection of functions which define classes of languages, which have the property that they are identifiable in the limit from positive data from a very simple kind of learner. Furthermore these learners are always incremental, maximally consistent, and locally conservative. They are also efficient provided the function itself is efficient. These learners are called string extension learners because components of the grammar are read directly from strings in the language via the defining function. A number of classes of languages in the literature can be described this way including varieties of kLocally Testable languages (McNaughton and Papert 1971) and kPiecewise Testable languages (Simon 1975), as well as some classes not discussed in the literature, such as the kPiecewise Testable languages in the Strict Sense. Potential applications of string extension learning exist for models of natural languages, particularly phonotactics, aspects of cognition and natural language processing.
Iterative Learning of Simple External Contextual Languages
"... Abstract. It is investigated for which choice of a parameter q, denoting the number of contexts, the class of simple external contextual languages is iteratively learnable. On one hand, the class admits, for all values of q, polynomial time learnability provided an adequate choice of the hypothesis ..."
Abstract

Cited by 9 (2 self)
 Add to MetaCart
(Show Context)
Abstract. It is investigated for which choice of a parameter q, denoting the number of contexts, the class of simple external contextual languages is iteratively learnable. On one hand, the class admits, for all values of q, polynomial time learnability provided an adequate choice of the hypothesis space is given. On the other hand, additional constraints like consistency and conservativeness or the use of a oneone hypothesis space changes the picture — iterative learning limits the long term memory of the learner to the current hypothesis and these constraints further hinder storage of information via padding of this hypothesis. It is shown that if q> 3, then simple external contextual languages are not iteratively learnable using a class preserving oneone hypothesis space, while for q = 1 it is iteratively learnable, even in polynomial time. It is also investigated for which choice of the parameters, the simple external contextual languages can be learnt by a consistent and conservative iterative learner. 1
On the Strength of Incremental Learning
, 1999
"... . This paper provides a systematic study of incremental learning from noisefree and from noisy data, thereby distinguishing between learning from only positive data and from both positive and negative data. Our study relies on the notion of noisy data introduced in [22]. The basic scenario, nam ..."
Abstract

Cited by 7 (4 self)
 Add to MetaCart
. This paper provides a systematic study of incremental learning from noisefree and from noisy data, thereby distinguishing between learning from only positive data and from both positive and negative data. Our study relies on the notion of noisy data introduced in [22]. The basic scenario, named iterative learning, is as follows. In every learning stage, an algorithmic learner takes as input one element of an information sequence for a target concept and its previously made hypothesis and outputs a new hypothesis. The sequence of hypotheses has to converge to a hypothesis describing the target concept correctly. We study the following refinements of this scenario. Bounded examplememory inference generalizes iterative inference by allowing an iterative learner to additionally store an a priori bounded number of carefully chosen data elements, while feedback learning generalizes it by allowing the iterative learner to additionally ask whether or not a particular data ele...
Ushaped learning may be necessary
"... Ushaped learning behaviour in cognitive development involves learning, unlearning and relearning. It occurs, for example, in learning irregular verbs. The prior cognitive science literature is occupied with how humans do it, for example, general rules versus tables of exceptions. This paper is mos ..."
Abstract

Cited by 7 (6 self)
 Add to MetaCart
Ushaped learning behaviour in cognitive development involves learning, unlearning and relearning. It occurs, for example, in learning irregular verbs. The prior cognitive science literature is occupied with how humans do it, for example, general rules versus tables of exceptions. This paper is mostly concerned with whether Ushaped learning behaviour may be necessary in the abstract mathematical setting of inductive inference, that is, in the computational learning theory following the framework of Gold. All notions considered are learning from text, that is, from positive data. Previous work showed that Ushaped learning behaviour is necessary for behaviourally correct learning but not for syntactically convergent, learning in the limit ( = explanatory learning). The present paper establishes the necessity for the whole hierarchy of classes of vacillatory learning where a behaviourally correct learner has to satisfy the additional constraint that it vacillates in the limit between at most k grammars, where k ≥ 1. Non Ushaped vacillatory learning is shown to be restrictive: Every non Ushaped vacillatorily learnable class is already learnable in the limit. Furthermore, if vacillatory learning with the parameter k = 2
Results on MemoryLimited UShaped Learning
"... Abstract. Ushaped learning is a learning behaviour in which the learner first learns a given target behaviour, then unlearns it and finally relearns it. Such a behaviour, observed by psychologists, for example, in the learning of pasttenses of English verbs, has been widely discussed among psychol ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
(Show Context)
Abstract. Ushaped learning is a learning behaviour in which the learner first learns a given target behaviour, then unlearns it and finally relearns it. Such a behaviour, observed by psychologists, for example, in the learning of pasttenses of English verbs, has been widely discussed among psychologists and cognitive scientists as a fundamental example of the nonmonotonicity of learning. Previous theory literature has studied whether or not Ushaped learning, in the context of Gold’s formal model of learning languages from positive data, is necessary for learning some tasks. It is clear that human learning involves memory limitations. In the present paper we consider, then, the question of the necessity of Ushaped learning for some learning models featuring memory limitations. Our results show that the question of the necessity of Ushaped learning in this memorylimited setting depends on delicate tradeoffs between the learner’s ability to remember its own previous conjecture, to store some values in its longterm memory, to make queries about whether or not items occur in previously seen data and on the learner’s choice of hypotheses space. 1
Non UShaped Vacillatory and Team Learning
, 2008
"... Ushaped learning behaviour in cognitive development involves learning, unlearning and relearning. It occurs, for example, in learning irregular verbs. The prior cognitive science literature is occupied with how humans do it, for example, general rules versus tables of exceptions. This paper is most ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
Ushaped learning behaviour in cognitive development involves learning, unlearning and relearning. It occurs, for example, in learning irregular verbs. The prior cognitive science literature is occupied with how humans do it, for example, general rules versus tables of exceptions. This paper is mostly concerned with whether Ushaped learning behaviour may be necessary in the abstract mathematical setting of inductive inference, that is, in the computational learning theory following the framework of Gold. All notions considered are learning from text, that is, from positive data. Previous work showed that Ushaped learning behaviour is necessary for behaviourally correct learning but not for syntactically convergent, learning in the limit ( = explanatory learning). The present paper establishes the necessity for the hierarchy of classes of vacillatory learning where a behaviourally correct learner has to satisfy the additional constraint that it vacillates in the limit between at most b grammars, where b ∈ {2, 3,...,∗}. Non Ushaped vacillatory learning is shown to be restrictive: every non Ushaped vacillatorily learnable class is already learnable in the limit. Furthermore, if vacillatory learning with the parameter b = 2 is possible then non Ushaped behaviourally correct learning is also possible. But for b = 3, surprisingly, there is a class witnessing that this implication fails.
An AverageCase Optimal OneVariable Pattern Language Learner
 Journal of Computer and System Sciences
, 2000
"... A new algorithm for learning onevariable pattern languages from positive data is proposed and analyzed with respect to its averagecase behavior. We consider the total learning time that takes into account all operations till convergence to a correct hypothesis is achieved. For almost all meaningfu ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
(Show Context)
A new algorithm for learning onevariable pattern languages from positive data is proposed and analyzed with respect to its averagecase behavior. We consider the total learning time that takes into account all operations till convergence to a correct hypothesis is achieved. For almost all meaningful distributions defining how the pattern variable is replaced by a string to generate random examples of the target pattern language, it is shown that this algorithm converges within an expected constant number of rounds and a total learning time that is linear in the pattern length. Thus, our solution is averagecase optimal in a strong sense. Though onevariable pattern languages can neither be finitely inferred from positive data nor PAClearned, our approach can also be extended to a probabilistic finite learner that exactly infers all onevariable pattern languages from positive data with high confidence. It is a long standing open problem whether pattern languages can be learned in...
A complete and tight averagecase analysis of learning monomials
 IN PROC. 16TH INT'L SYMPOS. ON THEORETICAL ASPECTS OF COMPUTER SCIENCE, STACS'99
, 1999
"... We advocate to analyze the average complexity of learning problems. An appropriate framework for this purpose is introduced. Based on it we consider the problem of learning monomials and the special case of learning monotone monomials in the limit and for online predictions in two variants: from ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
(Show Context)
We advocate to analyze the average complexity of learning problems. An appropriate framework for this purpose is introduced. Based on it we consider the problem of learning monomials and the special case of learning monotone monomials in the limit and for online predictions in two variants: from positive data only, and from positive and negative examples. The wellknown Wholist algorithm is completely analyzed, in particular its averagecase behavior with respect to the class of binomial distributions. We consider different complexity measures: the number of mind changes, the number of prediction errors, and the total learning time. Tight bounds are obtained implying that worst case bounds are too pessimistic. On the average learning can be achieved exponentially faster. Furthermore, we study a new learning model, stochastic finite learning, in which, in contrast to PAC learning, some information about the underlying distribution is given and the goal is to find a correct (not only approximatively correct) hypothesis. We develop techniques to obtain good bounds for stochastic finite learning from a precise average case analysis of strategies for learning in the limit and illustrate our approach for the case of learning monomials.
Strongly nonUshaped learning results by general techniques
 In Proc. of COLT’2010
, 2010
"... In learning, a semantic or behavioral Ushape occurs when a learner rst learns, then unlearns, and, nally, relearns, some target concept (on the way to success). Within the framework of Inductive Inference, previous results have shown, for example, that such Ushapes are unnecessary for explanatory ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
In learning, a semantic or behavioral Ushape occurs when a learner rst learns, then unlearns, and, nally, relearns, some target concept (on the way to success). Within the framework of Inductive Inference, previous results have shown, for example, that such Ushapes are unnecessary for explanatory learning, but are necessary for behaviorally correct and nontrivial vacillatory learning. Herein we focus more on syntactic Ushapes. This paper introduces two general techniques and applies them especially to syntactic Ushapes in learning: one technique to show when they are necessary and one to show when they are unnecessary. The technique for the former is very general and applicable to a much wider range of learning criteria. It employs socalled selflearning classes of languages which are shown to characterize completely one criterion learning more than another. We apply these techniques to show that, for setdriven and partially setdriven learning, any kind of Ushapes are unnecessary. Furthermore, we show that Ushapes are not unnecessary in a strong way for iterative learning, contrasting an earlier result by Case and Moelius that semantic Ushapes are unnecessary for iterative learning. 1