Results 1 
5 of
5
Mind Change Optimal Learning of Bayes Net Structure". O.Schulte
 in Proceedings of the 20th Annual Conference on Learning Theory
, 2007
"... Abstract. This paper analyzes the problem of learning the structure of a Bayes net (BN) in the theoretical framework of Gold’s learning paradigm. Bayes nets are one of the most prominent formalisms for knowledge representation and probabilistic and causal reasoning. We follow constraintbased approa ..."
Abstract

Cited by 9 (2 self)
 Add to MetaCart
Abstract. This paper analyzes the problem of learning the structure of a Bayes net (BN) in the theoretical framework of Gold’s learning paradigm. Bayes nets are one of the most prominent formalisms for knowledge representation and probabilistic and causal reasoning. We follow constraintbased approaches to learning Bayes net structure, where learning is based on observed conditional dependencies between variables of interest (e.g., “X is dependent on Y given any assignment to variable Z”). Applying learning criteria in this model leads to the following results. (1) The mind change complexity of identifying a Bayes net graph over variables V from dependency data is � � V, the maximum number of 2 edges. (2) There is a unique fastest mindchange optimal Bayes net learner; convergence speed is evaluated using Gold’s dominance notion of “uniformly faster convergence”. This learner conjectures a graph if it is the unique Bayes net pattern that satisfies the observed dependencies with a minimum number of edges, and outputs “no guess ” otherwise. Therefore we are using standard learning criteria to define a natural and novel Bayes net learning algorithm. We investigate the complexity of computing the output of the fastest mindchange optimal learner, and show that this problem is NPhard (assuming P=RP). To our knowledge this is the first NPhardness result concerning the existence of a uniquely optimal Bayes net structure. 1
Mind change efficient learning
 Info. & Comp
, 2005
"... Abstract. This paper studies efficient learning with respect to mind changes. Our starting point is the idea that a learner that is efficient with respect to mind changes minimizes mind changes not only globally in the entire learning problem, but also locally in subproblems after receiving some evi ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
Abstract. This paper studies efficient learning with respect to mind changes. Our starting point is the idea that a learner that is efficient with respect to mind changes minimizes mind changes not only globally in the entire learning problem, but also locally in subproblems after receiving some evidence. Formalizing this idea leads to the notion of uniform mind change optimality. We characterize the structure of language classes that can be identified with at most α mind changes by some learner (not necessarily effective): A language class L is identifiable with α mind changes iff the accumulation order of L is at most α. Accumulation order is a classic concept from pointset topology. To aid the construction of learning algorithms, we show that the characteristic property of uniformly mind change optimal learners is that they output conjectures (languages) with maximal accumulation order. We illustrate the theory by describing mind change optimal learners for various problems such as identifying linear subspaces and onevariable patterns. 1
Parsimony Hierarchies for Inductive Inference
 Journal of Symbolic Logic
"... Freivalds defined an acceptable programming system independent criterion for learning programs for functions in which the final programs were required to be both correct and "nearly" minimal size, i.e, within a computable function of being purely minimal size. Kinber showed that this parsimony requi ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Freivalds defined an acceptable programming system independent criterion for learning programs for functions in which the final programs were required to be both correct and "nearly" minimal size, i.e, within a computable function of being purely minimal size. Kinber showed that this parsimony requirement on final programs limits learning power. However, in scientific inference, parsimony is considered highly desirable. A limcomputable function is (by definition) one calculable by a total procedure allowed to change its mind finitely many times about its output. Investigated is the possibility of assuaging somewhat the limitation on learning power resulting from requiring parsimonious final programs by use of criteria which require the final, correct programs to be "notsonearly" minimal size, e.g., to be within a limcomputable function of actual minimal size. It is shown that some parsimony in the final program is thereby retained, yet learning power strictly increases. Considered, then, are limcomputable functions as above but for which notations for constructive ordinals are used to bound the number of mind changes allowed regarding the output. This is a variant of an idea introduced by Freivalds and Smith. For this ordinal notation complexity bounded version of limcomputability, the power of the resultant learning criteria form finely graded, infinitely ramifying, infinite hierarchies intermediate between the computable and the limcomputable cases. Some of these hierarchies, for the natural notations determining them, are shown to be optimally tight.
On a generalized notion of mistake bounds
 Information and Computation
"... This paper proposes the use of constructive ordinals as mistake bounds in the online learning model. This approach elegantly generalizes the applicability of the online mistake bound model to learnability analysis of very expressive concept classes like pattern languages, unions of pattern languag ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
This paper proposes the use of constructive ordinals as mistake bounds in the online learning model. This approach elegantly generalizes the applicability of the online mistake bound model to learnability analysis of very expressive concept classes like pattern languages, unions of pattern languages, elementary formal systems, and minimal models of logic programs. The main result in the paper shows that the topological property of effective finite bounded thickness is a sufficient condition for online learnability with a certain ordinal mistake bound. An interesting characterization of the online learning model is shown in terms of the identification in the limit framework. It is established that the classes of languages learnable in the online model with a mistake bound of α are exactly the same as the classes of languages learnable in the limit from both positive and negative data by a Popperian, consistent learner with a mind change bound of α. This result nicely builds a bridge between the two models. 1
Mind Change Optimal Learning: . . .
, 2007
"... Learning theories play a significant role to machine learning as computability and complexity theories to software engineering. Gold’s language learning paradigm is one cornerstone of modern learning theories. The aim of this thesis is to establish an inductive principle in Gold’s language learning ..."
Abstract
 Add to MetaCart
Learning theories play a significant role to machine learning as computability and complexity theories to software engineering. Gold’s language learning paradigm is one cornerstone of modern learning theories. The aim of this thesis is to establish an inductive principle in Gold’s language learning paradigm to guide the design of machine learning algorithms. We follow the common practice of using the number of mind changes to measure complexity of Gold’s language learning problems, and study efficient learning with respect to mind changes. Our starting point is the idea that a learner that is efficient with respect to mind changes minimizes mind changes not only globally in the entire learning problem, but also locally in subproblems after receiving some evidence. Formalizing this idea leads to the notion of mind change optimality. We characterize mind change complexity of language collections with Cantor’s classic concept of accumulation order. We show that the characteristic property of mind change optimal learners is that they output conjectures (languages) with maximal accumulation order. Therefore, we obtain an inductive principle in Gold’s language learning paradigm based on the simple topological concept accumulation order. The new