Results 1  10
of
35
Incremental concept learning for bounded data mining
 Information and Computation
, 1999
"... Important re nements of concept learning in the limit from positive data considerably restricting the accessibility of input data are studied. Let c be any concept; every in nite sequence of elements exhausting c is called positive presentation of c. In all learning models considered the learning ma ..."
Abstract

Cited by 42 (32 self)
 Add to MetaCart
(Show Context)
Important re nements of concept learning in the limit from positive data considerably restricting the accessibility of input data are studied. Let c be any concept; every in nite sequence of elements exhausting c is called positive presentation of c. In all learning models considered the learning machine computes a sequence of hypotheses about the target concept from a positive presentation of it. With iterative learning, the learning machine, in making a conjecture, has access to its previous conjecture and the latest data item coming in. In kbounded examplememory inference (k is a priori xed) the learner is allowed to access, in making a conjecture, its previous hypothesis, its memory of up to k data items it has already seen, and the next element coming in. In the case of kfeedback identi cation, the learning machine, in making a conjecture, has access to its previous conjecture, the latest data item coming in, and, on the basis of this information, it can compute k items and query the database of previous data to nd out, for each of the k items, whether or not it is in the database (k is again a priori xed). In all cases, the sequence of conjectures has to converge to a hypothesis
Incremental Learning using Sensitivity Analysis
, 1999
"... A new incremental learning algorithm for function approximation problems is presented where the neural network learner dynamically selects during training the most informative patterns from a candidate training set. The incremental learning algorithm uses its current knowledge about the function to ..."
Abstract

Cited by 12 (7 self)
 Add to MetaCart
A new incremental learning algorithm for function approximation problems is presented where the neural network learner dynamically selects during training the most informative patterns from a candidate training set. The incremental learning algorithm uses its current knowledge about the function to be approximated, in the form of output sensitivity information, to incrementally grow the training set with patterns that have the highest influence on the learning objective.
Iterative Learning of Simple External Contextual Languages
"... Abstract. It is investigated for which choice of a parameter q, denoting the number of contexts, the class of simple external contextual languages is iteratively learnable. On one hand, the class admits, for all values of q, polynomial time learnability provided an adequate choice of the hypothesis ..."
Abstract

Cited by 9 (2 self)
 Add to MetaCart
(Show Context)
Abstract. It is investigated for which choice of a parameter q, denoting the number of contexts, the class of simple external contextual languages is iteratively learnable. On one hand, the class admits, for all values of q, polynomial time learnability provided an adequate choice of the hypothesis space is given. On the other hand, additional constraints like consistency and conservativeness or the use of a oneone hypothesis space changes the picture — iterative learning limits the long term memory of the learner to the current hypothesis and these constraints further hinder storage of information via padding of this hypothesis. It is shown that if q> 3, then simple external contextual languages are not iteratively learnable using a class preserving oneone hypothesis space, while for q = 1 it is iteratively learnable, even in polynomial time. It is also investigated for which choice of the parameters, the simple external contextual languages can be learnt by a consistent and conservative iterative learner. 1
On the Strength of Incremental Learning
, 1999
"... . This paper provides a systematic study of incremental learning from noisefree and from noisy data, thereby distinguishing between learning from only positive data and from both positive and negative data. Our study relies on the notion of noisy data introduced in [22]. The basic scenario, nam ..."
Abstract

Cited by 7 (4 self)
 Add to MetaCart
. This paper provides a systematic study of incremental learning from noisefree and from noisy data, thereby distinguishing between learning from only positive data and from both positive and negative data. Our study relies on the notion of noisy data introduced in [22]. The basic scenario, named iterative learning, is as follows. In every learning stage, an algorithmic learner takes as input one element of an information sequence for a target concept and its previously made hypothesis and outputs a new hypothesis. The sequence of hypotheses has to converge to a hypothesis describing the target concept correctly. We study the following refinements of this scenario. Bounded examplememory inference generalizes iterative inference by allowing an iterative learner to additionally store an a priori bounded number of carefully chosen data elements, while feedback learning generalizes it by allowing the iterative learner to additionally ask whether or not a particular data ele...
Results on MemoryLimited UShaped Learning
"... Abstract. Ushaped learning is a learning behaviour in which the learner first learns a given target behaviour, then unlearns it and finally relearns it. Such a behaviour, observed by psychologists, for example, in the learning of pasttenses of English verbs, has been widely discussed among psychol ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
(Show Context)
Abstract. Ushaped learning is a learning behaviour in which the learner first learns a given target behaviour, then unlearns it and finally relearns it. Such a behaviour, observed by psychologists, for example, in the learning of pasttenses of English verbs, has been widely discussed among psychologists and cognitive scientists as a fundamental example of the nonmonotonicity of learning. Previous theory literature has studied whether or not Ushaped learning, in the context of Gold’s formal model of learning languages from positive data, is necessary for learning some tasks. It is clear that human learning involves memory limitations. In the present paper we consider, then, the question of the necessity of Ushaped learning for some learning models featuring memory limitations. Our results show that the question of the necessity of Ushaped learning in this memorylimited setting depends on delicate tradeoffs between the learner’s ability to remember its own previous conjecture, to store some values in its longterm memory, to make queries about whether or not items occur in previously seen data and on the learner’s choice of hypotheses space. 1
An AverageCase Optimal OneVariable Pattern Language Learner
 Journal of Computer and System Sciences
, 2000
"... A new algorithm for learning onevariable pattern languages from positive data is proposed and analyzed with respect to its averagecase behavior. We consider the total learning time that takes into account all operations till convergence to a correct hypothesis is achieved. For almost all meaningfu ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
(Show Context)
A new algorithm for learning onevariable pattern languages from positive data is proposed and analyzed with respect to its averagecase behavior. We consider the total learning time that takes into account all operations till convergence to a correct hypothesis is achieved. For almost all meaningful distributions defining how the pattern variable is replaced by a string to generate random examples of the target pattern language, it is shown that this algorithm converges within an expected constant number of rounds and a total learning time that is linear in the pattern length. Thus, our solution is averagecase optimal in a strong sense. Though onevariable pattern languages can neither be finitely inferred from positive data nor PAClearned, our approach can also be extended to a probabilistic finite learner that exactly infers all onevariable pattern languages from positive data with high confidence. It is a long standing open problem whether pattern languages can be learned in...
A complete and tight averagecase analysis of learning monomials
 IN PROC. 16TH INT'L SYMPOS. ON THEORETICAL ASPECTS OF COMPUTER SCIENCE, STACS'99
, 1999
"... We advocate to analyze the average complexity of learning problems. An appropriate framework for this purpose is introduced. Based on it we consider the problem of learning monomials and the special case of learning monotone monomials in the limit and for online predictions in two variants: from ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
(Show Context)
We advocate to analyze the average complexity of learning problems. An appropriate framework for this purpose is introduced. Based on it we consider the problem of learning monomials and the special case of learning monotone monomials in the limit and for online predictions in two variants: from positive data only, and from positive and negative examples. The wellknown Wholist algorithm is completely analyzed, in particular its averagecase behavior with respect to the class of binomial distributions. We consider different complexity measures: the number of mind changes, the number of prediction errors, and the total learning time. Tight bounds are obtained implying that worst case bounds are too pessimistic. On the average learning can be achieved exponentially faster. Furthermore, we study a new learning model, stochastic finite learning, in which, in contrast to PAC learning, some information about the underlying distribution is given and the goal is to find a correct (not only approximatively correct) hypothesis. We develop techniques to obtain good bounds for stochastic finite learning from a precise average case analysis of strategies for learning in the limit and illustrate our approach for the case of learning monomials.
Formal models of incremental learning and their analysis
"... Abstract — We consider concept learning from examples. The learner receives – step by step – larger and larger initial segments of a sequence of examples describing an unknown target concept, processes these examples, and computes hypotheses. The learner is successful, if its hypotheses stabilize on ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
Abstract — We consider concept learning from examples. The learner receives – step by step – larger and larger initial segments of a sequence of examples describing an unknown target concept, processes these examples, and computes hypotheses. The learner is successful, if its hypotheses stabilize on a correct representation of the target concept. The underlying model is called identification in the limit. The present study concerns different versions of incremental learning in the limit. In contrast to the general case, now the learner has only limited access to the examples provided so far. In the special case of iterative learning, the learner builds its new hypotheses just on the basis of the current hypothesis and the next example, without having access to any of the other examples presented so far. In the case of bounded examplememory learning, the learner may in addition memorize up to an a priori fixed number of examples already presented. Formal studies have shown that restricting the accessibility of the input data results in a loss of learning power, i. e. there are concept classes learnable in the limit, but not identifiable by any incremental learner at all. The present analysis aims at illustrating this phenomenon and giving insights into the structure of concept classes incremental learners can cope with. Examples of identifiable and nonidentifiable classes are given; different learning models are compared to one another with respect to the competence of the corresponding learners. I.
Sensitivity Analysis for Selective Learning by Feedforward Neural Networks
, 2001
"... Research on improving the performance of feedforward neural networks has concentrated mostly on the optimal setting of initial weights and learning parameters, sophisticated optimization techniques, architecture optimization, and adaptive activation functions. An alternative approach is presented in ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
Research on improving the performance of feedforward neural networks has concentrated mostly on the optimal setting of initial weights and learning parameters, sophisticated optimization techniques, architecture optimization, and adaptive activation functions. An alternative approach is presented in this paper where the neural network dynamically selects training patterns from a candidate training set during training, using the network's current attained knowledge about the target concept. Sensitivity analysis of the neural network output with respect to small input perturbations is used to quantify the informativeness of candidate patterns. Only the most informative patterns, which are those patterns closest to decision boundaries, are selected for training. Experimental results show a significant reduction in the training set size, without negatively influencing generalization performance and convergence characteristics. This approach to selective learning is then compared to an alternative where informativeness is measured as the magnitude in prediction error.
Can learning in the limit be done efficiently
 2842 in LNCS
, 2003
"... Abstract. Inductive inference can be considered as one of the fundamental paradigms of algorithmic learning theory. We survey results recently obtained and show their impact to potential applications. Since the main focus is put on the efficiency of learning, we also deal with postulates of natura ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Abstract. Inductive inference can be considered as one of the fundamental paradigms of algorithmic learning theory. We survey results recently obtained and show their impact to potential applications. Since the main focus is put on the efficiency of learning, we also deal with postulates of naturalness and their impact to the efficiency of limit learners. In particular, we look at the learnability of the class of all pattern languages and ask whether or not one can design a learner within the paradigm of learning in the limit that is nevertheless efficient. For achieving this goal, we deal with iterative learning and its interplay with the hypothesis spaces allowed. This interplay has also a severe impact to postulates of naturalness satisfiable by any learner. Finally, since a limit learner is only supposed to converge in the limit, one never knows at any particular learning stage whether or not the learner did already succeed. The resulting uncertainty may be prohibitive in many applications. We survey results to resolve this problem by outlining a new learning model, called stochastic finite learning. Though pattern languages can neither be finitely inferred from positive data nor PAClearned, our approach can be extended to a stochastic finite learner that exactly infers all pattern languages from positive data with high confidence. 1.