Results 1  10
of
38
Almost Everywhere High Nonuniform Complexity
, 1992
"... . We investigate the distribution of nonuniform complexities in uniform complexity classes. We prove that almost every problem decidable in exponential space has essentially maximum circuitsize and spacebounded Kolmogorov complexity almost everywhere. (The circuitsize lower bound actually exceeds ..."
Abstract

Cited by 169 (34 self)
 Add to MetaCart
. We investigate the distribution of nonuniform complexities in uniform complexity classes. We prove that almost every problem decidable in exponential space has essentially maximum circuitsize and spacebounded Kolmogorov complexity almost everywhere. (The circuitsize lower bound actually exceeds, and thereby strengthens, the Shannon 2 n n lower bound for almost every problem, with no computability constraint.) In exponential time complexity classes, we prove that the strongest relativizable lower bounds hold almost everywhere for almost all problems. Finally, we show that infinite pseudorandom sequences have high nonuniform complexity almost everywhere. The results are unified by a new, more powerful formulation of the underlying measure theory, based on uniform systems of density functions, and by the introduction of a new nonuniform complexity measure, the selective Kolmogorov complexity. This research was supported in part by NSF Grants CCR8809238 and CCR9157382 and in ...
Discovering Neural Nets With Low Kolmogorov Complexity And High Generalization Capability
 Neural Networks
, 1997
"... Many neural net learning algorithms aim at finding "simple" nets to explain training data. The expectation is: the "simpler" the networks, the better the generalization on test data (! Occam's razor). Previous implementations, however, use measures for "simplicity" that lack the power, universali ..."
Abstract

Cited by 49 (30 self)
 Add to MetaCart
Many neural net learning algorithms aim at finding "simple" nets to explain training data. The expectation is: the "simpler" the networks, the better the generalization on test data (! Occam's razor). Previous implementations, however, use measures for "simplicity" that lack the power, universality and elegance of those based on Kolmogorov complexity and Solomonoff's algorithmic probability. Likewise, most previous approaches (especially those of the "Bayesian" kind) suffer from the problem of choosing appropriate priors. This paper addresses both issues. It first reviews some basic concepts of algorithmic complexity theory relevant to machine learning, and how the SolomonoffLevin distribution (or universal prior) deals with the prior problem. The universal prior leads to a probabilistic method for finding "algorithmically simple" problem solutions with high generalization capability. The method is based on Levin complexity (a timebounded generalization of Kolmogorov comple...
Logical Depth and Physical Complexity
 THE UNIVERSAL TURING MACHINE: A HALFCENTURY SURVEY
, 1988
"... Some mathematical and natural objects (a random sequence, a sequence of zeros, a perfect crystal, a gas) are intuitively trivial, while others (e.g. the human body, the digits of #) contain internal evidence of a nontrivial causal history. We formalize this ..."
Abstract

Cited by 48 (0 self)
 Add to MetaCart
Some mathematical and natural objects (a random sequence, a sequence of zeros, a perfect crystal, a gas) are intuitively trivial, while others (e.g. the human body, the digits of #) contain internal evidence of a nontrivial causal history. We formalize this
Degrees of random sets
, 1991
"... An explicit recursiontheoretic definition of a random sequence or random set of natural numbers was given by MartinLöf in 1966. Other approaches leading to the notions of nrandomness and weak nrandomness have been presented by Solovay, Chaitin, and Kurtz. We investigate the properties of nrando ..."
Abstract

Cited by 46 (4 self)
 Add to MetaCart
An explicit recursiontheoretic definition of a random sequence or random set of natural numbers was given by MartinLöf in 1966. Other approaches leading to the notions of nrandomness and weak nrandomness have been presented by Solovay, Chaitin, and Kurtz. We investigate the properties of nrandom and weakly nrandom sequences with an emphasis on the structure of their Turing degrees. After an introduction and summary, in Chapter II we present several equivalent definitions of nrandomness and weak nrandomness including a new definition in terms of a forcing relation analogous to the characterization of ngeneric sequences in terms of Cohen forcing. We also prove that, as conjectured by Kurtz, weak nrandomness is indeed strictly weaker than nrandomness. Chapter III is concerned with intrinsic properties of nrandom sequences. The main results are that an (n + 1)random sequence A satisfies the condition A (n) ≡T A⊕0 (n) (strengthening a result due originally to Sacks) and that nrandom sequences satisfy a number of strong independence properties, e.g., if A ⊕ B is nrandom then A is nrandom relative to B. It follows that any countable distributive lattice can be embedded
Algorithmic Theories Of Everything
, 2000
"... The probability distribution P from which the history of our universe is sampled represents a theory of everything or TOE. We assume P is formally describable. Since most (uncountably many) distributions are not, this imposes a strong inductive bias. We show that P(x) is small for any universe x lac ..."
Abstract

Cited by 31 (15 self)
 Add to MetaCart
The probability distribution P from which the history of our universe is sampled represents a theory of everything or TOE. We assume P is formally describable. Since most (uncountably many) distributions are not, this imposes a strong inductive bias. We show that P(x) is small for any universe x lacking a short description, and study the spectrum of TOEs spanned by two Ps, one reflecting the most compact constructive descriptions, the other the fastest way of computing everything. The former derives from generalizations of traditional computability, Solomonoff’s algorithmic probability, Kolmogorov complexity, and objects more random than Chaitin’s Omega, the latter from Levin’s universal search and a natural resourceoriented postulate: the cumulative prior probability of all x incomputable within time t by this optimal algorithm should be 1/t. Between both Ps we find a universal cumulatively enumerable measure that dominates traditional enumerable measures; any such CEM must assign low probability to any universe lacking a short enumerating program. We derive Pspecific consequences for evolving observers, inductive reasoning, quantum physics, philosophy, and the expected duration of our universe.
Instance Complexity
, 1994
"... We introduce a measure for the computational complexity of individual instances of a decision problem and study some of its properties. The instance complexity of a string x with respect to a set A and time bound t, ic t (x : A), is defined as the size of the smallest specialcase program for A that ..."
Abstract

Cited by 29 (1 self)
 Add to MetaCart
We introduce a measure for the computational complexity of individual instances of a decision problem and study some of its properties. The instance complexity of a string x with respect to a set A and time bound t, ic t (x : A), is defined as the size of the smallest specialcase program for A that runs in time t, decides x correctly, and makes no mistakes on other strings ("don't know" answers are permitted). We prove that a set A is in P if and only if there exist a polynomial t and a constant c such that ic t (x : A) c for all x
On ResourceBounded Instance Complexity
 Theoretical Computer Science A
, 1995
"... The instance complexity of a string x with respect to a set A and time bound t, ic t (x : A), is the length of the shortest program for A that runs in time t, decides x correctly, and makes no mistakes on other strings (where "don't know" answers are permitted). The Instance Complexity Conjecture of ..."
Abstract

Cited by 19 (9 self)
 Add to MetaCart
The instance complexity of a string x with respect to a set A and time bound t, ic t (x : A), is the length of the shortest program for A that runs in time t, decides x correctly, and makes no mistakes on other strings (where "don't know" answers are permitted). The Instance Complexity Conjecture of Ko, Orponen, Schoning, and Watanabe states that for every recursive set A not in P and every polynomial t there is a polynomial t 0 and a constant c such that for infinitely many x, ic t (x : A) C t 0 (x) \Gamma c, where C t 0 (x) is the t 0 time bounded Kolmogorov complexity of x. In this paper the conjecture is proved for all recursive tally sets and for all recursive sets which are NPhard under honest reductions, in particular it holds for all natural NPhard problems. The method of proof also yields the polynomialspace bounded and the exponentialtime bounded versions of the conjecture in full generality. On the other hand, the conjecture itself turns out to be oracl...
Applications of TimeBounded Kolmogorov Complexity in Complexity Theory
 Kolmogorov complexity and computational complexity
, 1992
"... This paper presents one method of using timebounded Kolmogorov complexity as a measure of the complexity of sets, and outlines anumber of applications of this approach to di#erent questions in complexity theory. Connections will be drawn among the following topics: NE predicates, ranking functi ..."
Abstract

Cited by 18 (4 self)
 Add to MetaCart
This paper presents one method of using timebounded Kolmogorov complexity as a measure of the complexity of sets, and outlines anumber of applications of this approach to di#erent questions in complexity theory. Connections will be drawn among the following topics: NE predicates, ranking functions, pseudorandom generators, and hierarchy theorems in circuit complexity.
When Worlds Collide: Derandomization, Lower Bounds, and Kolmogorov Complexity
 OF REDUCTIONS,IN“PROC.29THACM SYMPOSIUM ON THEORY OF COMPUTING
, 1997
"... This paper has the following goals:  To survey some of the recent developments in the field of derandomization.  To introduce a new notion of timebounded Kolmogorov complexity (KT), and show that it provides a useful tool for understanding advances in derandomization, and for putting vario ..."
Abstract

Cited by 18 (5 self)
 Add to MetaCart
This paper has the following goals:  To survey some of the recent developments in the field of derandomization.  To introduce a new notion of timebounded Kolmogorov complexity (KT), and show that it provides a useful tool for understanding advances in derandomization, and for putting various results in context.  To illustrate the usefulness of KT, by answering a question that has been posed in the literature, and  To pose some promising directions for future research.