Abstract

Solomonoff completed the Bayesian framework by providing a rigorous, unique, formal, and universal choice for the model class and the prior. We discuss in breadth how and in which sense universal (non-i.i.d.) sequence prediction solves various (philosophical) problems of traditional Bayesian sequence prediction. We show that Solomonoff’s model possesses many desirable properties: Fast convergence and strong bounds, and in contrast to most classical continuous prior densities has no zero p(oste)rior problem, i.e. can confirm universal hypotheses, is reparametrization and regrouping invariant, and avoids the old-evidence and updating problem. It even performs well (actually better) in non-computable environments.

Citations

1412	An Introduction to Kolmogorov Complexity and Its Applications - Li, Vitanyi - 1997
316	A formal theory of inductive inference - Solomonoff - 1964
138	The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms - Zvonkin, Levin - 1970
121	Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability - Hutter - 2005
120	Clustering by compression - Cilibrasi, Vitanyi - 2005
111	The selection of prior distributions by formal rules - Kass, Wasserman - 1996
108	Complexity-Based Induction Systems: Comparisons and Convergence Theorems - Solomonoff - 1978
103	Inferences from multinomial data: learning about a bag of marbles - Walley - 1996
92	Information–theoretic asymptotics of Bayes methods - Clarke, Barron - 1990
69	or bust? A critical examination of Bayesian confirmation theory - Earman, “Bayes - 1992
67	Statistical and Inductive Inference by Minimum Message Length, Ed - Wallace - 2005
47	Optimal ordered problem solver - Schmidhuber - 2004
30	Hierarchies of generalized Kolmogorov complexities and nonenumerable universal measures computable in the limit - Schmidhuber
11	On the convergence speed of MDL predictions for Bernoulli sequences - Poland, Hutter - 2004