## Hierarchies Of Generalized Kolmogorov Complexities And Nonenumerable Universal Measures Computable In The Limit (2000)

Venue: | INTERNATIONAL JOURNAL OF FOUNDATIONS OF COMPUTER SCIENCE |

Citations: | 40 - 21 self |

### BibTeX

@ARTICLE{Schmidhuber00hierarchiesof,

author = {Jürgen Schmidhuber},

title = {Hierarchies Of Generalized Kolmogorov Complexities And Nonenumerable Universal Measures Computable In The Limit},

journal = {INTERNATIONAL JOURNAL OF FOUNDATIONS OF COMPUTER SCIENCE},

year = {2000},

volume = {13},

number = {4},

pages = {2002}

}

### Years of Citing Articles

### OpenURL

### Abstract

The traditional theory of Kolmogorov complexity and algorithmic probability focuses on monotone Turing machines with one-way write-only output tape. This naturally leads to the universal enumerable Solomono-Levin measure. Here we introduce more general, nonenumerable but cumulatively enumerable measures (CEMs) derived from Turing machines with lexicographically nondecreasing output and random input, and even more general approximable measures and distributions computable in the limit. We obtain a natural hierarchy of generalizations of algorithmic probability and Kolmogorov complexity, suggesting that the "true" information content of some (possibly in nite) bitstring x is the size of the shortest nonhalting program that converges to x and nothing but x on a Turing machine that can edit its previous outputs. Among other things we show that there are objects computable in the limit yet more random than Chaitin's "number of wisdom" Omega, that any approximable measure of x is small for any x lacking a short description, that there is no universal approximable distribution, that there is a universal CEM, and that any nonenumerable CEM of x is small for any x lacking a short enumerating program. We briey mention consequences for universes sampled from such priors.

### Citations

1782 | An Introduction to Kolmogorov Complexity and Its Applications, 2nd edn
- Li, Vitányi
- 1997
(Show Context)
Citation Context ...ralized Kolmogorov complexity hierarchy, halting probability Omega, cumulatively enumerable measures, computable universes. 1. Introduction and Outline We extend the theory of algorithmic probability =-=[40, 25, 13, 31, 41, 45, 28, 29, 16, 14, 17, 37, 1, 30, 39, 9, 42, 24, 23]-=- by studying nonenumerable priors computable in the limit [19, 33, 15]. This leads to a hierarchy of generalizations of the concepts of universal prior and Kolmogorov complexity. For instance, many ob... |

1305 |
On computable numbers with an application to the Entscheidungsproblem
- Turing
- 1936
(Show Context)
Citation Context ...er halting programs whose output is wrong for at mostsnitely many steps (Def. 2.11). This constructive notion of formal describability is less restrictive than the traditional notion of computability =-=[43]-=-, mainly because we do not insist on the existence of a halting program that computes an upper bound of the convergence time of the n-th bit of x. Formal describability thus pushes constructivism [5, ... |

421 |
Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme (On formally undecidable propositions in Principia Mathematica and related systems
- GÖDEL
- 1931
(Show Context)
Citation Context ...dditional bit, print \0". Aftersnite time the output will stabilize forever. Example 2.5 It is weakly decidable whether number theory is consistent (contrast this with Godel's famous negative res=-=ult [18]-=-). Theorem 2.1 (Convergence Problem) Given a GTM, it is not weakly decidable whether asnite bitstring is a converging program, or whether some of the output bits willsuctuate forever. Proof. (Based on... |

339 |
The definition of random sequences
- Martin-Löf
- 1966
(Show Context)
Citation Context ...extend T to x 2 B 1 , where we dene T (x) = T (x) = P T (x). If C denotes a set of TMs with universal element U C , then we write C (x) = U C (x); K C (x) := lg C (x) for C (x) > 0: (31) We observe that C is universal among all T-induced semimeasures, T 2 C. Note that C (x) = C (x0)+ C (x1)+P C (x) for x 2 B ; C (x) = P C (x) for x 2 B 1 : (32) It will be obvious from the ... |

337 | Algorithmic information theory
- Chaitin
- 1977
(Show Context)
Citation Context ...mpactly describable on EOMs but not on MTMs. To see this, consider Chaitin's the halting probability of an MTM whose input bits are obtained by tossing an unbiased coin whenever it requests a new bit =-=[14]-=-. is c.e. (dovetail over all programs p and sum up the contributions 2 l(p) of the halting p), but there is no recursive upper bound on the number of instructions required to compute n , given n. This... |

278 |
Foundations of Constructive Mathematics
- Beeson
- 1985
(Show Context)
Citation Context ...[43], mainly because we do not insist on the existence of a halting program that computes an upper bound of the convergence time of the n-th bit of x. Formal describability thus pushes constructivism =-=[5, 2-=-] to the extreme, barely avoiding the nonconstructivism embodied by even less restrictive concepts of describability | compare 0 n -describability [34]. Theorem 2.1 will generalize the weakly decidab... |

113 |
The concept of a random sequence
- Levin
- 1973
(Show Context)
Citation Context ...ralized Kolmogorov complexity hierarchy, halting probability Omega, cumulatively enumerable measures, computable universes. 1. Introduction and Outline We extend the theory of algorithmic probability =-=[40, 25, 13, 31, 41, 45, 28, 29, 16, 14, 17, 37, 1, 30, 39, 9, 42, 24, 23]-=- by studying nonenumerable priors computable in the limit [19, 33, 15]. This leads to a hierarchy of generalizations of the concepts of universal prior and Kolmogorov complexity. For instance, many ob... |

102 |
Trial and error predicates and solution to a problem of Mostowski
- Putnam
- 1965
(Show Context)
Citation Context ...duction and Outline We extend the theory of algorithmic probability [40, 25, 13, 31, 41, 45, 28, 29, 16, 14, 17, 37, 1, 30, 39, 9, 42, 24, 23] by studying nonenumerable priors computable in the limit =-=[19, 33, 15]-=-. This leads to a hierarchy of generalizations of the concepts of universal prior and Kolmogorov complexity. For instance, many objects with high traditional Kolmogorov complexity have low generalized... |

95 |
Limiting recursion
- Gold
- 1965
(Show Context)
Citation Context ...duction and Outline We extend the theory of algorithmic probability [40, 25, 13, 31, 41, 45, 28, 29, 16, 14, 17, 37, 1, 30, 39, 9, 42, 24, 23] by studying nonenumerable priors computable in the limit =-=[19, 33, 15]-=-. This leads to a hierarchy of generalizations of the concepts of universal prior and Kolmogorov complexity. For instance, many objects with high traditional Kolmogorov complexity have low generalized... |

94 |
On the symmetry of algorithmic information
- GÁCS
- 1974
(Show Context)
Citation Context ...ralized Kolmogorov complexity hierarchy, halting probability Omega, cumulatively enumerable measures, computable universes. 1. Introduction and Outline We extend the theory of algorithmic probability =-=[40, 25, 13, 31, 41, 45, 28, 29, 16, 14, 17, 37, 1, 30, 39, 9, 42, 24, 23]-=- by studying nonenumerable priors computable in the limit [19, 33, 15]. This leads to a hierarchy of generalizations of the concepts of universal prior and Kolmogorov complexity. For instance, many ob... |

87 |
Theory of Recursive Functions and Eective Computability
- Rogers
- 1967
(Show Context)
Citation Context ...mal describability thus pushes constructivism [5, 2] to the extreme, barely avoiding the nonconstructivism embodied by even less restrictive concepts of describability | compare 0 n -describability [=-=34]-=-. Theorem 2.1 will generalize the weakly decidable halting problem by demonstrating that it is not weakly decidable whether asnite string is a description of a describable object (there is a related i... |

67 | Randomness and recursive enumerability
- Kučera, Slaman
- 2001
(Show Context)
Citation Context |

55 |
A method for the construction of minimum-redundancy codes
- Human
- 1952
(Show Context)
Citation Context ...quality follows by denition. To show the right-hand side, one can build an EOM T that computes x 2 B ] using not more than KP E (x)+ K T (KP E (x))+O(1) input bits in a way inspired by Human-Coding [2=-=2]-=-. The claim then follows from the Invariance Theorem. The trick is to arrange T 's computation such that T 's output converges yet never needs to decrease lexicographically. T works as follows: (A) Em... |

41 | A Computer Scientist's View of Life, the Universe, and Everything
- Schmidhuber
- 1997
(Show Context)
Citation Context ...putes a never-ending sequence x ofsnite bitstrings x 1 ; x 2 ; : : : representing the history of some discrete universe, where x k represents the state at discrete time step k, and x 1 the \Big Bang&q=-=uot; [35]. \No-=-ise" will have to be simulated by invoking asnite pseudorandom generator subroutine PRG. Suppose its n-th output bit PRG(n) is 1 if the n-th program of a list of all possible programs halts, and ... |

40 |
On the length of programs for computing binary sequences
- Chaitin
- 1966
(Show Context)
Citation Context |

34 |
On the relation between descriptional complexity and algorithmic probability, Theor
- Gács
- 1983
(Show Context)
Citation Context ...er, and therefore is indeed EOM-computable. Obviously the following holds: CP T (x) = CP T (x) = C 0 (x) and P T (x) = X z2B ] P T (xz) = 0 (x): 2 Summary. The traditional universal c.e. measure [40,=-= 45, 29, 16, 17, 41, 14, 30]-=- derives from universal MTMs with random input. What is the nature of our novel generalization here? We simply replace the MTMs by EOMs. As shown above, this leads to universal cumulatively enumerable... |

34 |
A formal theory of inductive inference
- Solomono
(Show Context)
Citation Context |

33 |
A mathematical theory of communication, part i
- Shannon
- 1948
(Show Context)
Citation Context ...he complexity of the c.e. number 0:s 2 I(x) with minimal K T (s). The facts P x P M (x) = 1, P x P E (x) = 1, P x P G (x)s1, as well as intuition and wishful thinking inspired by Shannon-Fano Theorem =-=[38]-=- and Coding Theorem 5.1 suggest there might indeed be tighter bounds: Conjecture 5.1 For x 2 B ] with P M (x) > 0: P M (x) = O(2 K M (x) ). Conjecture 5.2 For x 2 B ] with P E (x) > 0: P E (x) = O(2 K... |

32 | Algorithmic Theories of Everything
- Schmidhuber
- 2000
(Show Context)
Citation Context ... is not [43]. Hence in general there is no computer that outputs x and only x without ever editing some previously computed history. That is, if we want to study the set of all programmable universes =-=[36]-=- then we should not limit ourselves to MTMs but consider GTMs as well. Note, however, that the output of a GTM might not stabilize in the sense that certain output bits mightsip from 0 to 1 and back f... |

23 |
of information (nongrowth) and aspects of the foundation of probability theory
- Levin
(Show Context)
Citation Context ...er, and therefore is indeed EOM-computable. Obviously the following holds: CP T (x) = CP T (x) = C 0 (x) and P T (x) = X z2B ] P T (xz) = 0 (x): 2 Summary. The traditional universal c.e. measure [40,=-= 45, 29, 16, 17, 41, 14, 30]-=- derives from universal MTMs with random input. What is the nature of our novel generalization here? We simply replace the MTMs by EOMs. As shown above, this leads to universal cumulatively enumerable... |

22 |
On the random oracle hypothesis
- Kurtz
- 1982
(Show Context)
Citation Context ...andom" than Chaitin's Omegas, in the sense that even on EOMs they do not have any compact descriptions | contrast this with Kurtz's and Kucera's nonconstructive randomness in the arithmetic hiera=-=rchy [2-=-7, 26]. Theorem 3.3 For all n there are z 2 B with K E (z) > n O(1); yet K G (z) O(log n): That is, K E (z) K G (z) is unbounded. Proof. For x 2 B nfg and universal EOM T dene (x) = X y2B ] :0:y>0:... |

19 |
Over de grondslagen der wiskunde
- Brouwer
(Show Context)
Citation Context ...[43], mainly because we do not insist on the existence of a halting program that computes an upper bound of the convergence time of the n-th bit of x. Formal describability thus pushes constructivism =-=[5, 2-=-] to the extreme, barely avoiding the nonconstructivism embodied by even less restrictive concepts of describability | compare 0 n -describability [34]. Theorem 2.1 will generalize the weakly decidab... |

15 |
Three approaches to the quantitative de of information. Problems of Information Transmission
- Kolmogorov
- 1965
(Show Context)
Citation Context |

14 | General loss bounds for universal sequence prediction
- Hutter
- 2001
(Show Context)
Citation Context |

12 |
On relative randomness
- Kučera
- 1993
(Show Context)
Citation Context ...andom" than Chaitin's Omegas, in the sense that even on EOMs they do not have any compact descriptions | contrast this with Kurtz's and Kucera's nonconstructive randomness in the arithmetic hiera=-=rchy [2-=-7, 26]. Theorem 3.3 For all n there are z 2 B with K E (z) > n O(1); yet K G (z) O(log n): That is, K E (z) K G (z) is unbounded. Proof. For x 2 B nfg and universal EOM T dene (x) = X y2B ] :0:y>0:... |

11 |
Process complexity and e�ective random tests� J.Comput
- Schnorr
(Show Context)
Citation Context |

10 |
On computable sequences
- Mostowski
- 1957
(Show Context)
Citation Context ...ite computation, there need not be a halting program that computes an upper bound of stabilization time, given any p and prex size. Compare the concept of computability in the limit [19, 33, 15] and [=-=20-=-, 32]. Denition 2.2 (TM-Specic Individual Describability) Given a TM T, an x 2 B ] is T-describable or T-computable i there is asnite p 2 B such that T (p) ; x. According to this denition, objects wi... |

9 | Analytic machines
- Hotz, Chadzelek
- 1997
(Show Context)
Citation Context ...bitstring is a converging program, or whether some of the output bits willsuctuate forever. Proof. (Based on the standard diagonalization trick [11, 18, 43]; compare a related result for analytic TMs =-=[12, 21]-=-). Let us write T (x) # if there is a z 2 B ] such that T (x) ; z. Let us write T (x) l if T 's outputsuctuates forever in response to x (e.g., bysipping from 1 to zero and back forever). Let A 1 ; A ... |

8 |
Recursively enumerable reals and Chaitin ! numbers
- Calude, Hertling, et al.
- 1998
(Show Context)
Citation Context ...encodable on EOMs than on traditional monotone TMs, and Theorem 3.3 shows that there are also objects with short GTM descriptions yet incompressible on EOMs and therefore \more random" than Chait=-=in's [14, 10, 39, 9, 42]-=-, the halting probability of a TM with random input, which is incompressible only on monotone TMs. This leads to a natural TM typespeci c Kolmogorov complexity hierarchy expressed by Inequality (12). ... |

8 |
Uber eine Eigenschaft des Inbegries aller reellen algebraischen Zahlen
- Cantor
(Show Context)
Citation Context ... Given a GTM, it is not weakly decidable whether asnite bitstring is a converging program, or whether some of the output bits willsuctuate forever. Proof. (Based on the standard diagonalization trick =-=[11, 18, 43]-=-; compare a related result for analytic TMs [12, 21]). Let us write T (x) # if there is a z 2 B ] such that T (x) ; z. Let us write T (x) l if T 's outputsuctuates forever in response to x (e.g., bysi... |

7 |
A version of O for which ZFC can not predict a single bit, In Finite Versus Infinite. Contributions to an Eternal a Dilemma; Calude
- Solovay
- 2000
(Show Context)
Citation Context |

5 |
Algorithmic information theory
- Barzdin
- 1988
(Show Context)
Citation Context |

5 |
Functions and functionals computable in the limit
- Freyvald
- 1977
(Show Context)
Citation Context ... 272 ] based on the work of Levin and Zvonkin [45]. Denition 4.1 (Semimeasures) A (binary) semimeasure is a function B ! [0; 1] that satises: () = 1; (x) 0; (x) = (x0) + (x1) + (x); (15) where is a function B ! [0; 1] satisfying 0 (x) (x). A notational dierence to the approach of Levin [45] (who writes (x) (x0) + (x1)) is the explicit introduction of . Compare ... |

5 |
New error bounds for Solomono prediction
- Hutter
- 1999
(Show Context)
Citation Context |

4 |
Quantum information and computation. Nature
- Bennett, DiVicenzo
- 2000
(Show Context)
Citation Context ...isc, or a never ending evolution of a virtual reality determined by asnite algorithm, such as in Example 2.1? Contrary to a widely spread misunderstanding, quantum physics, quantum computation (e.g., =-=[3]-=-) and Heisenberg's uncertainty principle do not rule this out [35]. In absence of contrarian evidence we might assume our universe is formally describable indeed, or at least sampled from a formally d... |

3 |
Inductive Turing machines. Notices of the Academy of Sciences of the USSR (translated from Russian
- Burgin
- 1991
(Show Context)
Citation Context ...niversal Turing Machines (TMs) more general than those considered in previous related work: unlike traditional TMs, General TMs or GTMs may edit their previous outputs (compare Burgin's inductive TMs =-=[6-=-]), and Enumerable Output Machines (EOMs) may do this provided the output does not decrease lexicographically. In the spirit of computability in the limit [19, 33, 15] we will dene: a formally describ... |

3 |
Infinite processes and super-recursive algorithms
- Burgin, Borodyanskii
- 1991
(Show Context)
Citation Context ...d of the output tape (if it is not the initial square or above the scanning head); (c) write 1 or 0 on square above output scanning head. Compare Burgin's inductive TMs and super-recursive algorithms =-=[6, 8]-=-. Example 2.1 (Pseudorandom universe based on halting problem) Why consider GTMs at all? Because in a certain sense they represent the non-plus-ultra of constructive computability. For instance, consi... |

3 |
Complexity-based induction systems
- Solomono
- 1978
(Show Context)
Citation Context ...are selected randomly. Since some of the possible input sequences cause nonhalting computations, the individual probabilities do not add up to 1. This and related issues inspired work on semimeasures =-=[40, 45, 41, 17, 3-=-0] as opposed to classical measures considered in traditional statistics. The next three denitions concerning semimeasures on B are almost but not quite identical to those of discrete semimeasures [3... |

2 |
Chaitin\Omega numbers, Solovay machines and Godel incompleteness. Theoretical Computer Science
- Calude
- 2000
(Show Context)
Citation Context |

2 |
The complexity of objects and the algorithmic concepts of information and randomness
- Zvonkin, Levin
(Show Context)
Citation Context |

1 |
Theory of super-recursive algorithms as a source of a new paradigm for computer simulation
- Burgin
- 2000
(Show Context)
Citation Context ... GTMs may occasionally rewrite parts of their output, they are computationally more expressive than MTMs in the sense that they permit much more compact descriptions of certain objects | compare also =-=[7-=-]. For instance, K(x) K G (x) is unbounded, as will be shown next. This will later have consequences for predictions, given certain observations. Theorem 3.2 K(x) K G (x) is unbounded. Proof. Dene h 0... |

1 |
On the de of computable real continuous functions. Fundamenta Mathematicae
- Gregorczyk
- 1957
(Show Context)
Citation Context ...ite computation, there need not be a halting program that computes an upper bound of stabilization time, given any p and prex size. Compare the concept of computability in the limit [19, 33, 15] and [=-=20-=-, 32]. Denition 2.2 (TM-Specic Individual Describability) Given a TM T, an x 2 B ] is T-describable or T-computable i there is asnite p 2 B such that T (p) ; x. According to this denition, objects wi... |

1 |
Non-stochastic in and sequences
- V'yugin
- 1998
(Show Context)
Citation Context ...eralized Kolmogorov Complexity for EOMs and GTMs Denition 3.2 (Generalized K T ) Given any TM T, dene K T (x) = min p fl(p) : T (p) ; xg Compare Schnorr's less general \process complexity" for MT=-=Ms [37, 44-=-]. Proposition 3.1 (K M ; K E ; K G based on Invariance Theorem) Consider Def. 2.4. Let C denote a set of TMs with universal TM U C (T 2 C). We drop the index T , writing K C (x) = K U C (x) K T (x) ... |