Discovering Problem Solutions with Low Kolmogorov Complexity and High Generalization Capability (1994)
| Venue: | MACHINE LEARNING: PROCEEDINGS OF THE TWELFTH INTERNATIONAL CONFERENCE |
| Citations: | 17 - 9 self |
BibTeX
@TECHREPORT{Schmidhuber94discoveringproblem,
author = {Jürgen Schmidhuber},
title = {Discovering Problem Solutions with Low Kolmogorov Complexity and High Generalization Capability},
institution = {MACHINE LEARNING: PROCEEDINGS OF THE TWELFTH INTERNATIONAL CONFERENCE},
year = {1994}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
Many machine learning algorithms aim at finding "simple" rules to explain training data. The expectation is: the "simpler" the rules, the better the generalization on test data (! Occam's razor). Most practical implementations, however, use measures for "simplicity" that lack the power, universality and elegance of those based on Kolmogorov complexity and Solomonoff's algorithmic probability. Likewise, most previous approaches (especially those of the "Bayesian" kind) suffer from the problem of choosing appropriate priors. This paper addresses both issues. It first reviews some basic concepts of algorithmic complexity theory relevant to machine learning, and how the Solomonoff-Levin distribution (or universal prior) deals with the prior problem. The universal prior leads to a probabilistic method for finding "algorithmically simple" problem solutions with high generalization capability. The method is based on Levin complexity (a time-bounded generalization of Kolmogorov complexity) and...
