Results 1  10
of
27
Learning indexed families of recursive languages from positive data: a survey
, 2008
"... In the past 40 years, research on inductive inference has developed along different lines, e.g., in the formalizations used, and in the classes of target concepts considered. One common root of many of these formalizations is Gold’s model of identification in the limit. This model has been studied f ..."
Abstract

Cited by 14 (6 self)
 Add to MetaCart
In the past 40 years, research on inductive inference has developed along different lines, e.g., in the formalizations used, and in the classes of target concepts considered. One common root of many of these formalizations is Gold’s model of identification in the limit. This model has been studied for learning recursive functions, recursively enumerable languages, and recursive languages, reflecting different aspects of machine learning, artificial intelligence, complexity theory, and recursion theory. One line of research focuses on indexed families of recursive languages — classes of recursive languages described in a representation scheme for which the question of membership for any string in any of the given languages is effectively decidable with a uniform procedure. Such language classes are of interest because of their naturalness. The survey at hand picks out important studies on learning indexed families (including basic as well as recent research), summarizes and illustrates the corresponding results, and points out links to related fields such as grammatical inference, machine learning, and artificial intelligence in general.
String Extension Learning
, 2009
"... This paper defines a collection of functions which define classes of languages, which have the property that they are identifiable in the limit from positive data from a very simple kind of learner. Furthermore these learners are always incremental, maximally consistent, and locally conservative. Th ..."
Abstract

Cited by 8 (4 self)
 Add to MetaCart
This paper defines a collection of functions which define classes of languages, which have the property that they are identifiable in the limit from positive data from a very simple kind of learner. Furthermore these learners are always incremental, maximally consistent, and locally conservative. They are also efficient provided the function itself is efficient. These learners are called string extension learners because components of the grammar are read directly from strings in the language via the defining function. A number of classes of languages in the literature can be described this way including varieties of kLocally Testable languages (McNaughton and Papert 1971) and kPiecewise Testable languages (Simon 1975), as well as some classes not discussed in the literature, such as the kPiecewise Testable languages in the Strict Sense. Potential applications of string extension learning exist for models of natural languages, particularly phonotactics, aspects of cognition and natural language processing.
On the Strength of Incremental Learning
, 1999
"... . This paper provides a systematic study of incremental learning from noisefree and from noisy data, thereby distinguishing between learning from only positive data and from both positive and negative data. Our study relies on the notion of noisy data introduced in [22]. The basic scenario, nam ..."
Abstract

Cited by 7 (4 self)
 Add to MetaCart
. This paper provides a systematic study of incremental learning from noisefree and from noisy data, thereby distinguishing between learning from only positive data and from both positive and negative data. Our study relies on the notion of noisy data introduced in [22]. The basic scenario, named iterative learning, is as follows. In every learning stage, an algorithmic learner takes as input one element of an information sequence for a target concept and its previously made hypothesis and outputs a new hypothesis. The sequence of hypotheses has to converge to a hypothesis describing the target concept correctly. We study the following refinements of this scenario. Bounded examplememory inference generalizes iterative inference by allowing an iterative learner to additionally store an a priori bounded number of carefully chosen data elements, while feedback learning generalizes it by allowing the iterative learner to additionally ask whether or not a particular data ele...
Non UShaped Vacillatory and Team Learning
, 2008
"... Ushaped learning behaviour in cognitive development involves learning, unlearning and relearning. It occurs, for example, in learning irregular verbs. The prior cognitive science literature is occupied with how humans do it, for example, general rules versus tables of exceptions. This paper is most ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
Ushaped learning behaviour in cognitive development involves learning, unlearning and relearning. It occurs, for example, in learning irregular verbs. The prior cognitive science literature is occupied with how humans do it, for example, general rules versus tables of exceptions. This paper is mostly concerned with whether Ushaped learning behaviour may be necessary in the abstract mathematical setting of inductive inference, that is, in the computational learning theory following the framework of Gold. All notions considered are learning from text, that is, from positive data. Previous work showed that Ushaped learning behaviour is necessary for behaviourally correct learning but not for syntactically convergent, learning in the limit ( = explanatory learning). The present paper establishes the necessity for the hierarchy of classes of vacillatory learning where a behaviourally correct learner has to satisfy the additional constraint that it vacillates in the limit between at most b grammars, where b ∈ {2, 3,...,∗}. Non Ushaped vacillatory learning is shown to be restrictive: every non Ushaped vacillatorily learnable class is already learnable in the limit. Furthermore, if vacillatory learning with the parameter b = 2 is possible then non Ushaped behaviourally correct learning is also possible. But for b = 3, surprisingly, there is a class witnessing that this implication fails.
Results on MemoryLimited UShaped Learning
"... Abstract. Ushaped learning is a learning behaviour in which the learner first learns a given target behaviour, then unlearns it and finally relearns it. Such a behaviour, observed by psychologists, for example, in the learning of pasttenses of English verbs, has been widely discussed among psychol ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
Abstract. Ushaped learning is a learning behaviour in which the learner first learns a given target behaviour, then unlearns it and finally relearns it. Such a behaviour, observed by psychologists, for example, in the learning of pasttenses of English verbs, has been widely discussed among psychologists and cognitive scientists as a fundamental example of the nonmonotonicity of learning. Previous theory literature has studied whether or not Ushaped learning, in the context of Gold’s formal model of learning languages from positive data, is necessary for learning some tasks. It is clear that human learning involves memory limitations. In the present paper we consider, then, the question of the necessity of Ushaped learning for some learning models featuring memory limitations. Our results show that the question of the necessity of Ushaped learning in this memorylimited setting depends on delicate tradeoffs between the learner’s ability to remember its own previous conjecture, to store some values in its longterm memory, to make queries about whether or not items occur in previously seen data and on the learner’s choice of hypotheses space. 1
Iterative Learning of Simple External Contextual Languages
"... Abstract. It is investigated for which choice of a parameter q, denoting the number of contexts, the class of simple external contextual languages is iteratively learnable. On one hand, the class admits, for all values of q, polynomial time learnability provided an adequate choice of the hypothesis ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
Abstract. It is investigated for which choice of a parameter q, denoting the number of contexts, the class of simple external contextual languages is iteratively learnable. On one hand, the class admits, for all values of q, polynomial time learnability provided an adequate choice of the hypothesis space is given. On the other hand, additional constraints like consistency and conservativeness or the use of a oneone hypothesis space changes the picture — iterative learning limits the long term memory of the learner to the current hypothesis and these constraints further hinder storage of information via padding of this hypothesis. It is shown that if q> 3, then simple external contextual languages are not iteratively learnable using a class preserving oneone hypothesis space, while for q = 1 it is iteratively learnable, even in polynomial time. It is also investigated for which choice of the parameters, the simple external contextual languages can be learnt by a consistent and conservative iterative learner. 1
A complete and tight averagecase analysis of learning monomials
 IN PROC. 16TH INT'L SYMPOS. ON THEORETICAL ASPECTS OF COMPUTER SCIENCE, STACS'99
, 1999
"... We advocate to analyze the average complexity of learning problems. An appropriate framework for this purpose is introduced. Based on it we consider the problem of learning monomials and the special case of learning monotone monomials in the limit and for online predictions in two variants: from ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
We advocate to analyze the average complexity of learning problems. An appropriate framework for this purpose is introduced. Based on it we consider the problem of learning monomials and the special case of learning monotone monomials in the limit and for online predictions in two variants: from positive data only, and from positive and negative examples. The wellknown Wholist algorithm is completely analyzed, in particular its averagecase behavior with respect to the class of binomial distributions. We consider different complexity measures: the number of mind changes, the number of prediction errors, and the total learning time. Tight bounds are obtained implying that worst case bounds are too pessimistic. On the average learning can be achieved exponentially faster. Furthermore, we study a new learning model, stochastic finite learning, in which, in contrast to PAC learning, some information about the underlying distribution is given and the goal is to find a correct (not only approximatively correct) hypothesis. We develop techniques to obtain good bounds for stochastic finite learning from a precise average case analysis of strategies for learning in the limit and illustrate our approach for the case of learning monomials.
An AverageCase Optimal OneVariable Pattern Language Learner
 Journal of Computer and System Sciences
, 2000
"... A new algorithm for learning onevariable pattern languages from positive data is proposed and analyzed with respect to its averagecase behavior. We consider the total learning time that takes into account all operations till convergence to a correct hypothesis is achieved. For almost all meaningfu ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
A new algorithm for learning onevariable pattern languages from positive data is proposed and analyzed with respect to its averagecase behavior. We consider the total learning time that takes into account all operations till convergence to a correct hypothesis is achieved. For almost all meaningful distributions defining how the pattern variable is replaced by a string to generate random examples of the target pattern language, it is shown that this algorithm converges within an expected constant number of rounds and a total learning time that is linear in the pattern length. Thus, our solution is averagecase optimal in a strong sense. Though onevariable pattern languages can neither be finitely inferred from positive data nor PAClearned, our approach can also be extended to a probabilistic finite learner that exactly infers all onevariable pattern languages from positive data with high confidence. It is a long standing open problem whether pattern languages can be learned in...
Variants of Iterative Learning
, 1998
"... We investigate the principal learning capabilities of iterative learners in some more details. Thereby, we confine ourselves to study the learnability of indexable concept classes. The general scenario of iterative learning is as follows. An iterative learner successively takes as input one element ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
We investigate the principal learning capabilities of iterative learners in some more details. Thereby, we confine ourselves to study the learnability of indexable concept classes. The general scenario of iterative learning is as follows. An iterative learner successively takes as input one element of a text (an informant) for a target concept as well as its previously made hypothesis and outputs a new hypothesis about the target concept. The sequence of hypotheses has to converge to a hypothesis correctly describing the target concept.
Parsimony Hierarchies for Inductive Inference
 Journal of Symbolic Logic
"... Freivalds defined an acceptable programming system independent criterion for learning programs for functions in which the final programs were required to be both correct and "nearly" minimal size, i.e, within a computable function of being purely minimal size. Kinber showed that this parsimony requi ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Freivalds defined an acceptable programming system independent criterion for learning programs for functions in which the final programs were required to be both correct and "nearly" minimal size, i.e, within a computable function of being purely minimal size. Kinber showed that this parsimony requirement on final programs limits learning power. However, in scientific inference, parsimony is considered highly desirable. A limcomputable function is (by definition) one calculable by a total procedure allowed to change its mind finitely many times about its output. Investigated is the possibility of assuaging somewhat the limitation on learning power resulting from requiring parsimonious final programs by use of criteria which require the final, correct programs to be "notsonearly" minimal size, e.g., to be within a limcomputable function of actual minimal size. It is shown that some parsimony in the final program is thereby retained, yet learning power strictly increases. Considered, then, are limcomputable functions as above but for which notations for constructive ordinals are used to bound the number of mind changes allowed regarding the output. This is a variant of an idea introduced by Freivalds and Smith. For this ordinal notation complexity bounded version of limcomputability, the power of the resultant learning criteria form finely graded, infinitely ramifying, infinite hierarchies intermediate between the computable and the limcomputable cases. Some of these hierarchies, for the natural notations determining them, are shown to be optimally tight.