. We investigate the distribution of nonuniform complexities in uniform complexity classes. We prove that almost every problem decidable in exponential space has essentially maximum circuit-size and space-bounded Kolmogorov complexity almost everywhere. (The circuit-size lower bound actually exceeds, and thereby strengthens, the Shannon 2 n n lower bound for almost every problem, with no computability constraint.) In exponential time complexity classes, we prove that the strongest relativizable lower bounds hold almost everywhere for almost all problems. Finally, we show that infinite pseudorandom sequences have high nonuniform complexity almost everywhere. The results are unified by a new, more powerful formulation of the underlying measure theory, based on uniform systems of density functions, and by the introduction of a new nonuniform complexity measure, the selective Kolmogorov complexity. This research was supported in part by NSF Grants CCR-8809238 and CCR-9157382 and in ...

Abstract not found

Many neural net learning algorithms aim at finding "simple" nets to explain training data. The expectation is: the "simpler" the networks, the better the generalization on test data (! Occam's razor). Previous implementations, however, use measures for "simplicity" that lack the power, universality and elegance of those based on Kolmogorov complexity and Solomonoff's algorithmic probability. Likewise, most previous approaches (especially those of the "Bayesian" kind) suffer from the problem of choosing appropriate priors. This paper addresses both issues. It first reviews some basic concepts of algorithmic complexity theory relevant to machine learning, and how the Solomonoff-Levin distribution (or universal prior) deals with the prior problem. The universal prior leads to a probabilistic method for finding "algorithmically simple" problem solutions with high generalization capability. The method is based on Levin complexity (a time-bounded generalization of Kolmogorov comple...

Some mathematical and natural objects (a random sequence, a sequence of zeros, a perfect crystal, a gas) are intuitively trivial, while others (e.g. the human body, the digits of #) contain internal evidence of a nontrivial causal history. We formalize this

An explicit recursion-theoretic definition of a random sequence or random set of natural numbers was given by Martin-Löf in 1966. Other approaches leading to the notions of n-randomness and weak n-randomness have been presented by Solovay, Chaitin, and Kurtz. We investigate the properties of n-random and weakly n-random sequences with an emphasis on the structure of their Turing degrees. After an introduction and summary, in Chapter II we present several equivalent definitions of n-randomness and weak n-randomness including a new definition in terms of a forcing relation analogous to the characterization of n-generic sequences in terms of Cohen forcing. We also prove that, as conjectured by Kurtz, weak nrandomness is indeed strictly weaker than n-randomness. Chapter III is concerned with intrinsic properties of n-random sequences. The main results are that an (n + 1)-random sequence A satisfies the condition A (n) ≡T A⊕0 (n) (strengthening a result due originally to Sacks) and that n-random sequences satisfy a number of strong independence properties, e.g., if A ⊕ B is n-random then A is n-random relative to B. It follows that any countable distributive lattice can be embedded

We introduce a measure for the computational complexity of individual instances of a decision problem and study some of its properties. The instance complexity of a string x with respect to a set A and time bound t, ic t (x : A), is defined as the size of the smallest special-case program for A that runs in time t, decides x correctly, and makes no mistakes on other strings ("don't know" answers are permitted). We prove that a set A is in P if and only if there exist a polynomial t and a constant c such that ic t (x : A) c for all x

The probability distribution P from which the history of our universe is sampled represents a theory of everything or TOE. We assume P is formally describable. Since most (uncountably many) distributions are not, this imposes a strong inductive bias. We show that P(x) is small for any universe x lacking a short description, and study the spectrum of TOEs spanned by two Ps, one reflecting the most compact constructive descriptions, the other the fastest way of computing everything. The former derives from generalizations of traditional computability, Solomonoff’s algorithmic probability, Kolmogorov complexity, and objects more random than Chaitin’s Omega, the latter from Levin’s universal search and a natural resource-oriented postulate: the cumulative prior probability of all x incomputable within time t by this optimal algorithm should be 1/t. Between both Ps we find a universal cumulatively enumerable measure that dominates traditional enumerable measures; any such CEM must assign low probability to any universe lacking a short enumerating program. We derive P-specific consequences for evolving observers, inductive reasoning, quantum physics, philosophy, and the expected duration of our universe.

The instance complexity of a string x with respect to a set A and time bound t, ic t (x : A), is the length of the shortest program for A that runs in time t, decides x correctly, and makes no mistakes on other strings (where "don't know" answers are permitted). The Instance Complexity Conjecture of Ko, Orponen, Schoning, and Watanabe states that for every recursive set A not in P and every polynomial t there is a polynomial t 0 and a constant c such that for infinitely many x, ic t (x : A) C t 0 (x) \Gamma c, where C t 0 (x) is the t 0 -time bounded Kolmogorov complexity of x. In this paper the conjecture is proved for all recursive tally sets and for all recursive sets which are NP-hard under honest reductions, in particular it holds for all natural NP-hard problems. The method of proof also yields the polynomial-space bounded and the exponential-time bounded versions of the conjecture in full generality. On the other hand, the conjecture itself turns out to be oracl...

This paper presents one method of using time-bounded Kolmogorov complexity as a measure of the complexity of sets, and outlines anumber of applications of this approach to di#erent questions in complexity theory. Connections will be drawn among the following topics: NE predicates, ranking functions, pseudorandom generators, and hierarchy theorems in circuit complexity.

This paper has the following goals: -- To survey some of the recent developments in the field of derandomization. -- To introduce a new notion of time-bounded Kolmogorov complexity (KT), and show that it provides a useful tool for understanding advances in derandomization, and for putting various results in context. -- To illustrate the usefulness of KT, by answering a question that has been posed in the literature, and -- To pose some promising directions for future research.