## Incremental concept learning for bounded data mining (1999)

### Cached

### Download Links

Venue: | Information and Computation |

Citations: | 39 - 29 self |

### BibTeX

@ARTICLE{Jain99incrementalconcept,

author = {Sanjay Jain and Steffen Lange and Thomas Zeugmann},

title = {Incremental concept learning for bounded data mining},

journal = {Information and Computation},

year = {1999}

}

### Years of Citing Articles

### OpenURL

### Abstract

Important re nements of concept learning in the limit from positive data considerably restricting the accessibility of input data are studied. Let c be any concept; every in nite sequence of elements exhausting c is called positive presentation of c. In all learning models considered the learning machine computes a sequence of hypotheses about the target concept from a positive presentation of it. With iterative learning, the learning machine, in making a conjecture, has access to its previous conjecture and the latest data item coming in. In k-bounded example-memory inference (k is a priori xed) the learner is allowed to access, in making a conjecture, its previous hypothesis, its memory of up to k data items it has already seen, and the next element coming in. In the case of k-feedback identi cation, the learning machine, in making a conjecture, has access to its previous conjecture, the latest data item coming in, and, on the basis of this information, it can compute k items and query the database of previous data to nd out, for each of the k items, whether or not it is in the database (k is again a priori xed). In all cases, the sequence of conjectures has to converge to a hypothesis

### Citations

893 | Language identification in the limit - Gold - 1967 |

837 | Theory of Recursive Functions and Effective Computability - Rogers - 1967 |

630 |
From data mining to knowledge discovery: an overview
- Fayyad, Piatetsky-Shapiro, et al.
- 1996
(Show Context)
Citation Context ...he data. Thus, the additional steps such as data presentation, data selection, incorporating prior knowledge, and de ning the semantics of the results obtained belong to KDD (cf., e.g., Fayyad et al. =-=[14]-=-). Prominent examples of KDD applications in health care and nance include Matheus et al. [27] and Kloesgen [22]. The importance of KDD research nds its explanation in the fact that the data collected... |

296 |
Inductive inference of formal languages from positive data
- Angluin
- 1978
(Show Context)
Citation Context ...-program i. Hence, we also refer to the index i of Wi as a grammar for Wi. Furthermore, we sometimes consider the scenario that indexed families of recursive languages have to be learned (cf. Angluin =-=[2]-=-). Let be any nite alphabet of symbols, and let X be the free monoid over , i.e., X = . As usual, we refer to subsets L X as to languages. A class of non-empty recursive languages L is said to be an i... |

285 |
Formal Languages and Their Relation to Automata
- Hopcroft, Ullman
- 1969
(Show Context)
Citation Context ...feedback machine can decide which k items to lookup to see if they are in the database. There are apparent 1 The sub-focus on learning grammars, or, equivalently, recognizers (cf. Hopcroft and Ullman =-=[19]-=-), for concepts from positive instances nicely models the situation where the database ags or contains examples of the concept to be learned and doesn't ag or contain the non-examples. 2That the conce... |

259 |
Toward a mathematical theory of inductive inference
- Blum, Blum
- 1975
(Show Context)
Citation Context ...ges to be in TxtEx Pat (2 ). Wright [44] extended this result to PAT (k) 2 TxtEx Pat (k ) for all k 1. Moreover, Theorem 4.2 in Shinohara and Arimura's [40] together with a lemma from Blum and Blum's =-=[7]-=- shows that S k2IN PAT (k) is not TxtEx H-inferable for every hypothesis space H. However, nothing was known previous to the present paper concerning the incremental learnability of PAT (k). We resolv... |

218 |
Systems that Learn: An Introduction to Learning Theory, second edition
- Jain, Osherson, et al.
- 1999
(Show Context)
Citation Context ... and TxtFex-inference. The assertion remains true even if the learner is only allowed to vacillate between up to 2 descriptions, i.e., in the case jDj 2 (cf. Case [9, 11]). Theorem 1 (Osherson et al. =-=[31]-=-; Case [9, 11]). T xtEx a T xtF ex a , for all a 2 IN[f g. Looking at the above de nitions, we see that an IIM M has always access to the whole history of the learning process, i.e., in order to compu... |

208 |
Finding patterns common to a set of strings
- Angluin
- 1980
(Show Context)
Citation Context ...em 7 is the most di cult for the present paper. Theorem 8 shows that, even in fairly concrete contexts, with iterative learning, redundancy in the hypothesis space increases learning power. Angluin's =-=[1]-=- pattern languages are learnable from positive data, and they (and nite unions thereof) have been extensively studied and applied to molecular biology and to the learning of interesting special classe... |

163 | Comparison of identification criteria for machine inductive inference, Theoretical Computer Science - Case, Smith - 1983 |

146 |
The Process of Knowledge Discovery in Databases: A Human-Centered Approach
- Brachman, Anand
- 1996
(Show Context)
Citation Context ... concerns the choice of the right sampling size. One way proposed to solve this problem as well as other problems related to huge data sets is interaction and iteration (cf., e.g., Brachman and Anand =-=[6]-=- and Fayyad et al. [14]). That is, the whole data mining process is iterated a few times, thereby allowing human interaction until a satisfactory interpretation of the data is found. Looking at data m... |

74 | A machine independent theory of the complexity of recursive functions - Blum - 1967 |

69 |
Theory of Formal Systems
- SMULLYAN
- 1962
(Show Context)
Citation Context ...for solving problems in molecular biology (cf., e.g. Shimozono et al. [36], Shinohara and Arikawa [39]). Pattern languages and nite unions of pattern languages turn out to be subclasses of Smullyan's =-=[41]-=-) elementary formal systems (EFS). Arikawa et al. [3] have shown that EFS can also be treated as a logic programming language over strings. Recently, the techniques for learning nite unions of pattern... |

62 |
Editing by example
- Nix
- 1985
(Show Context)
Citation Context ...Salomaa [34, 35], and Shinohara and Arikawa [39] for an overview). Moreover, Angluin [1] proved that the class of all pattern languages is learnable in the limit from positive data. Subsequently, Nix =-=[30]-=- as well as Shinohara and Arikawa [39] outlined interesting applications of pattern inference algorithms. For example, pattern language learning algorithms have been successfully applied for solving p... |

56 | A guided tour across the boundaries of learning recursive languages
- Zeugmann, Lange
(Show Context)
Citation Context ...nd all strings x 2X wehave f(j; x) = ( 1; if x 2 Lj; 0; otherwise: Since the paper of Angluin [2] learning of indexed families of languages has attracted much attention (cf., e.g., Zeugmann and Lange =-=[46]-=-). Mainly, this seems due to the fact that most of the established language families such as regular languages, context-free languages, context-sensitive languages, and pattern languages are indexed f... |

53 |
N.Weir,” Automating the Analysis and cataloging of Sky Su rveys
- Fayyad, Djorgovski
- 1996
(Show Context)
Citation Context ...tions, knowledge, patterns and the like from huge collections of data. Usually, the data are available from massive data sets collected, for example, by scienti c instruments (cf., e.g.,Fayyad et al. =-=[13]-=-), by scientists all over the world (as in the human genome project), or in databases that have been built for other purposes than a current purpose. We shall be mainly concerned with the extraction o... |

49 |
Periodicity in generations of automata
- Case
- 1974
(Show Context)
Citation Context ... Mi+1( )=M(Mi();xi+1). For any nite sequence , let ProgSet(M ; )=f 1(M ( )) g, and we de ne for any text T the set ProgSet(M ;T) similarly. Then by implicit use of the Operator Recursion Theorem (cf. =-=[8, 10]-=-) there exists a recursive 1{1 increasing function p, such that Wp( ) may be de ned as follows (p(0) will be nice). Enumerate h0;p(0)i in Wp(0). Let 0 be such that content( 0) = fh0;p(0)ig. Let W s p(... |

44 | The power of vacillation in language learning
- Case
- 1999
(Show Context)
Citation Context ...and for two reasons: 1. It is arguable that all natural languages are in nite, and 2. many language learning unsolvability results depend strongly on including the nite languages (cf. Gold [18]; Case =-=[11]-=-). Ditto for other results below, namely, Theorems 6 and 7, which are witnessed by concept classes containing only in nite concepts.4 similarities between these two kinds of learning machines, yet Th... |

41 |
Limeserkennung Rekursiver Funktionen Durch Spezielle Stratgien Elektronische Informationsverarbeitung und Kybernetik
- Wiehagen
- 1976
(Show Context)
Citation Context ...oncept for computing its actual guess. Conceptionally, an iterative IIMM de nes a sequence (M n) n2N of machines each of which takes as its input the output of its predecessor. Definition 3 (Wiehagen =-=[42]-=-). Let C be a concept class, let c be a concept, let H = (hj) j2IN be a hypothesis space, and let a 2 IN [ f g. An IIM M TxtItEx a H {infers c i for every T =(xj) j2IN2text(c) the following conditions... |

38 |
Polynomial-time inference of arbitrary pattern languages
- Lange, Wiehagen
- 1991
(Show Context)
Citation Context ... ! L 6 L(Q0 )]g. Club stands for consistent least upper bounds. As already mentioned above, the class PAT is TxtEx Pat-learnable from positive data (cf. Angluin [1]). Subsequently, Lange and Wiehagen =-=[23]-=- showed PAT to be TxtItEx Patinferable. As for unions, the rst result goes back to Shinohara [37] who proved the class of all unions of at most two pattern languages to be in TxtEx Pat (2 ). Wright [4... |

38 | Knowledge acquisition from amino acid sequences by machine learning system BONSAI
- Shimozono, Shinohara, et al.
- 1994
(Show Context)
Citation Context ...ng applications of pattern inference algorithms. For example, pattern language learning algorithms have been successfully applied for solving problems in molecular biology (cf., e.g. Shimozono et al. =-=[36]-=-, Shinohara and Arikawa [39]). Pattern languages and nite unions of pattern languages turn out to be subclasses of Smullyan's [41]) elementary formal systems (EFS). Arikawa et al. [3] have shown that ... |

37 |
Theory of Recursive Functions and E ective Computability
- Rogers
- 1967
(Show Context)
Citation Context ... for each k>0, the concept class consisting of all unions of at most k pattern languages is learnable (from positive data) by an iterative machine! 2. Preliminaries Unspeci ed notation follows Rogers =-=[33]-=-. In addition to or in contrast with Rogers [33] we use the following. By IN = f0; 1; 2;:::g we denote the set of all natural numbers. We set IN + =INnf0g. The cardinality of a set S is denoted by jSj... |

34 |
On the role of procrastination in machine learning
- Freivalds, Smith
- 1993
(Show Context)
Citation Context ...sions of the incremental learning models considered. A natural relaxation of the constraint to x k a priori can be obtained by using the notion of constructive ordinals as done by Freivalds and Smith =-=[16]-=- for mind changes. Intuitively, the paramenter k is now speci ed to be a constructive ordinal, and the bounded examplememory learner as well as a feedback machine can change their mind of how many dat... |

32 | Incremental learning from positive data
- Lange, Zeugmann
- 1996
(Show Context)
Citation Context ... as input the output of its predecessor. Consequently, a bounded example-memory IIM has to output a hypothesis as well as a subset of the set of examples seen so far. Definition 4 (Lange and Zeugmann =-=[25]-=-). Let k 2 IN, let C be aconcept class, let c be a concept, let H =(hj)j2INbeahypothesis space, and let a 2 IN[f g. An IIM M TxtBem k Ex a H { infers c i for every T =(xj) j2IN2text(c) the following c... |

30 |
The power of vacillation
- Case
- 1969
(Show Context)
Citation Context ... [18] classical learning in the limit and TxtFex-inference. The assertion remains true even if the learner is only allowed to vacillate between up to 2 descriptions, i.e., in the case jDj 2 (cf. Case =-=[9, 11]-=-). Theorem 1 (Osherson et al. [31]; Case [9, 11]). T xtEx a T xtF ex a , for all a 2 IN[f g. Looking at the above de nitions, we see that an IIM M has always access to the whole history of the learnin... |

30 | Identification of unions of languages drawn from an identifiable class, in - Wright - 1989 |

26 |
Inductive inference of monotonic formal systems from positive data. New Generation Computing 8 (4):371–384. Special Issue on Algorithmic Learning Theory for ALT’90
- Shinohara
- 1991
(Show Context)
Citation Context ...logic programming language over strings. Recently, the techniques for learning nite unions of pattern languages have been extended to show the learnability of various subclasses of EFS (cf. Shinohara =-=[38]-=-). From a theoretical point of view, investigations of the learnability of subclasses of EFS are important because they yield corresponding results about the learnability of subclasses of logic progra... |

20 | Characterizations of monotonic and dual monotonic language learning
- Zeugmann, Lange, et al.
- 1995
(Show Context)
Citation Context ...ecial interest, and future research should address the problem under what circumstances which model is preferable. Characterizations may serve as suitable tool for accomplishing this goal (cf., e.g., =-=[2, 7, 47]-=-). Additionally, feed-back identi cation and bounded example-memory inference have been considered in the general context of classes of recursively enumerable concepts rather than uniformly recursives... |

19 | Ingnoring data may be the only way to learn efficiently - Wiehagen, Zeugmann - 1994 |

18 | Infinitary self-reference in learning theory
- Case
- 1994
(Show Context)
Citation Context ... Mi+1( )=M(Mi();xi+1). For any nite sequence , let ProgSet(M ; )=f 1(M ( )) g, and we de ne for any text T the set ProgSet(M ;T) similarly. Then by implicit use of the Operator Recursion Theorem (cf. =-=[8, 10]-=-) there exists a recursive 1{1 increasing function p, such that Wp( ) may be de ned as follows (p(0) will be nice). Enumerate h0;p(0)i in Wp(0). Let 0 be such that content( 0) = fh0;p(0)ig. Let W s p(... |

18 | Efficient discovery of interesting statements in databases - Kloesgen - 1995 |

18 |
Selecting and reporting what is interesting
- Matheus, Piatetsky-Shapiro, et al.
(Show Context)
Citation Context ...rior knowledge, and de ning the semantics of the results obtained belong to KDD (cf., e.g., Fayyad et al. [14]). Prominent examples of KDD applications in health care and nance include Matheus et al. =-=[27]-=- and Kloesgen [22]. The importance of KDD research nds its explanation in the fact that the data collected in various elds such as biology, nance, retail, astronomy, medicine are extremely rapidly gro... |

18 |
Pattern Inference
- Shinohara, Arikawa
- 1995
(Show Context)
Citation Context ...Pattern Languages The pattern languages (de ned two paragraphs below) were formally introduced by Angluin [1] and have been widely investigated (cf., e.g., Salomaa [34, 35], and Shinohara and Arikawa =-=[39]-=- for an overview). Moreover, Angluin [1] proved that the class of all pattern languages is learnable in the limit from positive data. Subsequently, Nix [30] as well as Shinohara and Arikawa [39] outli... |

16 | Inclusion is undecidable for pattern languages - Jiang, Salomaa, et al. - 1993 |

14 | Set-driven and rearrangement-independent learning of recursive languages - Lange, Zeugmann - 1996 |

13 | Learning recursive languages with bounded mind changes
- Lange, Zeugmann
- 1993
(Show Context)
Citation Context ... canonical index in ~H for T Cons." We leave it to the reader to verify that ~ M witnesses L 2 TxtEx ~H . Finally, the TxtEx Hinferability ofLdirectly follows from Proposition 1 in Lange and Zeugmann =-=[24]-=-, and thus Lemma 1 is proved. Lemma 2. Let L be an indexed family exclusively containing in nite languages such that L2TxtEx . Then there are a hypothesis space H =(h j) j2INand an IIM M such that (1)... |

12 |
Inductive inference of unbounded unions of pattern languages from positive data
- Shinohara, Arimura
- 1996
(Show Context)
Citation Context ...lass of all unions of at most two pattern languages to be in TxtEx Pat (2 ). Wright [44] extended this result to PAT (k) 2 TxtEx Pat (k ) for all k 1. Moreover, Theorem 4.2 in Shinohara and Arimura's =-=[40]-=- together with a lemma from Blum and Blum's [7] shows that S k2IN PAT (k) is not TxtEx H-inferable for every hypothesis space H. However, nothing was known previous to the present paper concerning the... |

11 | Inductive inference of Prolog programs with linear data dependency from positive data
- Arimura, Shinohara
- 1994
(Show Context)
Citation Context ...t of view, investigations of the learnability of subclasses of EFS are important because they yield corresponding results about the learnability of subclasses of logic programs. Arimura and Shinohara =-=[4]-=- have used the insight gained from the learnability of EFS subclasses to show that a class of linearly covering logic programs with local variables is identi able in the limit from only positive data.... |

10 |
Return to patterns (the formal language theory column
- Salomaa
- 1995
(Show Context)
Citation Context ...k;0i. We omit the details. 3.4. The Pattern Languages The pattern languages (de ned two paragraphs below) were formally introduced by Angluin [1] and have been widely investigated (cf., e.g., Salomaa =-=[34, 35]-=-, and Shinohara and Arikawa [39] for an overview). Moreover, Angluin [1] proved that the class of all pattern languages is learnable in the limit from positive data. Subsequently, Nix [30] as well as ... |

8 |
Open problems in ”systems that learn
- Fulk, Jain, et al.
- 1994
(Show Context)
Citation Context ..., Mn(T ) is de ned, where M0(T )=dfM(x0)and for all n 0: Mn+1(T )=dfM(Mn(T);An k (Qk(Mn(T);xn+1));xn+1), 4 Our de nition is a variant of one found in Osherson, Stob and Weinstein [31] and Fulk et al. =-=[17]-=- which will be discussed later.8 (2) the sequence (Mn(T ))n2IN converges to a number j such that c = a hj provided that An k truthfully answers the questions computed by Qk (i.e. the j-th component o... |

8 |
Inferring unions of two pattern languages
- Shinohara
- 1983
(Show Context)
Citation Context ...lass PAT is TxtEx Pat-learnable from positive data (cf. Angluin [1]). Subsequently, Lange and Wiehagen [23] showed PAT to be TxtItEx Patinferable. As for unions, the rst result goes back to Shinohara =-=[37]-=- who proved the class of all unions of at most two pattern languages to be in TxtEx Pat (2 ). Wright [44] extended this result to PAT (k) 2 TxtEx Pat (k ) for all k 1. Moreover, Theorem 4.2 in Shinoha... |

7 | On monotonic strategies for learning r.e. languages - Jain, Sharma - 1994 |

6 |
A class of Prolog programs inferable from positive data
- Rao
- 1996
(Show Context)
Citation Context ... EFS subclasses to show that a class of linearly covering logic programs with local variables is identi able in the limit from only positive data. More recently, using similar techniques, Krishna-Rao =-=[32]-=- has established the learnability from only positive data of an even larger class of logic programs. These results have consequences for Inductive Logic Programming. 6 Patterns and pattern languages a... |

6 | and Wiehagen's pattern language learning algorithm: An averagecase analysis with respect to its total learning time, Annals of Mathematics and Arti cial Intelligence
- Zeugmann, Lange
- 1997
(Show Context)
Citation Context ...., the overall time needed until convergence is exponential in the number of di erent variables occurring in the target pattern for inputs drawn with respect to the uniform distribution (cf. Zeugmann =-=[45]-=-). Second, we considerably generalized the model of feedback inference introduced in [25] byREFERENCES 29 allowing the feedback learner to ask simultaneously k queries. Though at rst glance it may se... |

6 |
Comparison of identi cation criteria for machine inductive inference, Theoretical Computer Science 25
- Case, Smith
- 1983
(Show Context)
Citation Context ...sses C for which there are an IIM M and a hypothesis space H such that M TxtEx a H -infers C.6 The a represents the number of mistakes or anomalies allowed in the nal conjectures (cf. Case and Smith =-=[12]-=-), with a =0being Gold's [18] original case where no mistakes are allowed. If a =0,weusually omit the upper index, e.g., we write TxtEx instead of TxtEx 0 . We adopt this convention in the de nitions ... |

6 | Language identi cation - Gold - 1967 |

5 | The power of vacillation, in "Proceedings 1st Workshop on Computational Learning Theory - Case - 1988 |

5 |
The relation of two patterns with comparable languages
- File
- 1988
(Show Context)
Citation Context ...e study so-called non-erasing substitutions. It is also possible to consider erasing substitutions where variables may be replaced by empty strings, leading to a di erent class of languages (cf. File =-=[15]-=-).25 (2) The sequence Club(L 0;k); Club(L 1;k); ::: converges to Club(L; k). Proof. (1): Fix k 1, and suppose T = s0;s1:::;sn;sn+1;:::is a text for L. Furthermore, let S0 = ff g s0 2 L( )g. We procee... |

5 |
Probabilistic language learning under monotonicity constraints
- Meyer
- 1997
(Show Context)
Citation Context ...lso be interesting to extend this and the topics of the present paper to probabilistic learning machines. This branch of learning theory has recently seen as variety of surprising results (cf., e.g., =-=[20, 28, 29]-=-), and thus, one may expect further interesting insight into the power of probabilism by combining it with incremental learning. Finally, while the research presented in the present paper clari ed wha... |

4 |
Learning elementary formal systems, Theoretical Computer Science 95
- Arikawa, Shinohara, et al.
- 1992
(Show Context)
Citation Context ...himozono et al. [36], Shinohara and Arikawa [39]). Pattern languages and nite unions of pattern languages turn out to be subclasses of Smullyan's [41]) elementary formal systems (EFS). Arikawa et al. =-=[3]-=- have shown that EFS can also be treated as a logic programming language over strings. Recently, the techniques for learning nite unions of pattern languages have been extended to show the learnabilit... |

4 | On identification by teams and probabilistic machines - Jain, Sharma - 1995 |