Results 1  10
of
57
PACLearnability of Determinate Logic Programs
, 1992
"... The field of Inductive Logic Programming (ILP) is concerned with inducing logic programs from examples in the presence of background knowledge. This paper defines the ILP problem, and describes the various syntactic restrictions that are commonly used for learning firstorder representations. We the ..."
Abstract

Cited by 75 (8 self)
 Add to MetaCart
The field of Inductive Logic Programming (ILP) is concerned with inducing logic programs from examples in the presence of background knowledge. This paper defines the ILP problem, and describes the various syntactic restrictions that are commonly used for learning firstorder representations. We then derive some positive results concerning the learnability of these restricted classes of logic programs, by reducing the ILP problem to a standard propositional learning problem. More specifically, kclause predicate definitions consisting of determinate, functionfree, nonrecursive Horn clauses with variables of bounded depth are polynomially learnable under a broad class of probability distributions, called simple distributions. Similarly, recursive kclause definitions are polynomially learnable under simple distributions if we allow existential and membership queries about the target concept.
Discovering Neural Nets With Low Kolmogorov Complexity And High Generalization Capability
 Neural Networks
, 1997
"... Many neural net learning algorithms aim at finding "simple" nets to explain training data. The expectation is: the "simpler" the networks, the better the generalization on test data (! Occam's razor). Previous implementations, however, use measures for "simplicity&quo ..."
Abstract

Cited by 54 (31 self)
 Add to MetaCart
(Show Context)
Many neural net learning algorithms aim at finding "simple" nets to explain training data. The expectation is: the "simpler" the networks, the better the generalization on test data (! Occam's razor). Previous implementations, however, use measures for "simplicity" that lack the power, universality and elegance of those based on Kolmogorov complexity and Solomonoff's algorithmic probability. Likewise, most previous approaches (especially those of the "Bayesian" kind) suffer from the problem of choosing appropriate priors. This paper addresses both issues. It first reviews some basic concepts of algorithmic complexity theory relevant to machine learning, and how the SolomonoffLevin distribution (or universal prior) deals with the prior problem. The universal prior leads to a probabilistic method for finding "algorithmically simple" problem solutions with high generalization capability. The method is based on Levin complexity (a timebounded generalization of Kolmogorov comple...
Learning Regular Languages From Simple Positive Examples
, 2000
"... Learning from positive data constitutes an important topic in Grammatical Inference since it is believed that the acquisition of grammar by children only needs syntactically correct (i.e. positive) instances. However, classical learning models provide no way to avoid the problem of overgeneralizati ..."
Abstract

Cited by 29 (0 self)
 Add to MetaCart
(Show Context)
Learning from positive data constitutes an important topic in Grammatical Inference since it is believed that the acquisition of grammar by children only needs syntactically correct (i.e. positive) instances. However, classical learning models provide no way to avoid the problem of overgeneralization. In order to overcome this problem, we use here a learning model from simple examples, where the notion of simplicity is defined with the help of Kolmogorov complexity. We show that a general and natural heuristic which allows learning from simple positive examples can be developed in this model. Our main result is that the class of regular languages is probably exactly learnable from simple positive examples.
Towards a universal theory of artificial intelligence based on algorithmic probability and sequential decisions
 Proceedings of the 12 th Eurpean Conference on Machine Learning (ECML2001
, 2001
"... Abstract. Decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental probability distribution is known. Solomonoff’s theory of universal induction formally solves the problem of sequence prediction for unknown distributions. We unify both theories an ..."
Abstract

Cited by 28 (10 self)
 Add to MetaCart
Abstract. Decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental probability distribution is known. Solomonoff’s theory of universal induction formally solves the problem of sequence prediction for unknown distributions. We unify both theories and give strong arguments that the resulting universal AIξ model behaves optimally in any computable environment. The major drawback of the AIξ model is that it is uncomputable. To overcome this problem, we construct a modified algorithm AIξ tl, which is still superior to any other time t and length l bounded agent. The computation time of AIξ tl is of the order t·2 l. 1
Grammar Inference, Automata Induction, and Language Acquisition
 Handbook of Natural Language Processing
, 2000
"... The natural language learning problem has attracted the attention of researchers for several decades. Computational and formal models of language acquisition have provided some preliminary, yet promising insights of how children learn the language of their community. Further, these formal models als ..."
Abstract

Cited by 27 (2 self)
 Add to MetaCart
(Show Context)
The natural language learning problem has attracted the attention of researchers for several decades. Computational and formal models of language acquisition have provided some preliminary, yet promising insights of how children learn the language of their community. Further, these formal models also provide an operational framework for the numerous practical applications of language learning. We will survey some of the key results in formal language learning. In particular, we will discuss the prominent computational approaches for learning different classes of formal languages and discuss how these fit in the broad context of natural language learning.
A Theory of Universal Artificial Intelligence based on Algorithmic Complexity
, 2000
"... Decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental prior probability distribution is known. Solomonoff's theory of universal induction formally solves the problem of sequence prediction for unknown prior distribution. We combine both ide ..."
Abstract

Cited by 26 (11 self)
 Add to MetaCart
(Show Context)
Decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental prior probability distribution is known. Solomonoff's theory of universal induction formally solves the problem of sequence prediction for unknown prior distribution. We combine both ideas and get a parameterless theory of universal Artificial Intelligence. We give strong arguments that the resulting AIXI model is the most intelligent unbiased agent possible. We outline for a number of problem classes, including sequence prediction, strategic games, function minimization, reinforcement and supervised learning, how the AIXI model can formally solve them. The major drawback of the AIXI model is that it is uncomputable. To overcome this problem, we construct a modified algorithm AIXItl, which is still effectively more intelligent than any other time t and space l bounded agent. The computation time of AIXItl is of the order t·2^l. Other discussed topics are formal definitions of intelligence order relations, the horizon problem and relations of the AIXI theory to other AI approaches.
Learning DFA from Simple Examples
, 1997
"... Efficient learning of DFA is a challenging research problem in grammatical inference. It is known that both exact and approximate (in the PAC sense) identifiability of DFA is hard. Pitt, in his seminal paper posed the following open research problem: "Are DFAPACidentifiable if examples are d ..."
Abstract

Cited by 22 (5 self)
 Add to MetaCart
(Show Context)
Efficient learning of DFA is a challenging research problem in grammatical inference. It is known that both exact and approximate (in the PAC sense) identifiability of DFA is hard. Pitt, in his seminal paper posed the following open research problem: "Are DFAPACidentifiable if examples are drawn from the uniform distribution, or some other known simple distribution?" [25]. We demonstrate that the class of simple DFA (i.e., DFA whose canonical representations have logarithmic Kolmogorov complexity) is efficiently PAC learnable under the Solomonoff Levin universal distribution. We prove that if the examples are sampled at random according to the universal distribution by a teacher that is knowledgeable about the target concept, the entire class of DFA is efficiently PAC learnable under the universal distribution. Thus, we show that DFA are efficiently learnable under the PACS model [6]. Further, we prove that any concept that is learnable under Gold's model for learning from characteristic samples, Goldman and Mathias' polynomial teachability model, and the model for learning from example based queries is also learnable under the PACS model.
Learning logic programs with structured background knowledge (Extended Abstract)
"... The polynomial PAC  learnability of nonrecursive Horn clauses is studied, based on a characterization of the least general generalization of a set of positive examples in terms of products and homomorphisms. This approach is used to show that nonrecursive Horn clauses are polynomially PAC  learna ..."
Abstract

Cited by 21 (5 self)
 Add to MetaCart
The polynomial PAC  learnability of nonrecursive Horn clauses is studied, based on a characterization of the least general generalization of a set of positive examples in terms of products and homomorphisms. This approach is used to show that nonrecursive Horn clauses are polynomially PAC  learnable if there is a single binary background predicate and the ground facts in the background knowledge form a forest. If the ground facts in the background knowledge form a disjoint union of cycles then the situation is different as the shortest consistent hypothesis may have exponential length. In this case polynomial PAC  learnability holds if a different concept representation is used. We also consider the learnability of multiple clauses in some restricted cases. 1 Introduction The theoretical study of efficient learnability developed into a separate field of research in the last decade, motivated by the increasing importance of learning in practical applications. Several learning probl...