Results 1  10
of
66
Almost Everywhere High Nonuniform Complexity
, 1992
"... . We investigate the distribution of nonuniform complexities in uniform complexity classes. We prove that almost every problem decidable in exponential space has essentially maximum circuitsize and spacebounded Kolmogorov complexity almost everywhere. (The circuitsize lower bound actually exceeds ..."
Abstract

Cited by 170 (34 self)
 Add to MetaCart
. We investigate the distribution of nonuniform complexities in uniform complexity classes. We prove that almost every problem decidable in exponential space has essentially maximum circuitsize and spacebounded Kolmogorov complexity almost everywhere. (The circuitsize lower bound actually exceeds, and thereby strengthens, the Shannon 2 n n lower bound for almost every problem, with no computability constraint.) In exponential time complexity classes, we prove that the strongest relativizable lower bounds hold almost everywhere for almost all problems. Finally, we show that infinite pseudorandom sequences have high nonuniform complexity almost everywhere. The results are unified by a new, more powerful formulation of the underlying measure theory, based on uniform systems of density functions, and by the introduction of a new nonuniform complexity measure, the selective Kolmogorov complexity. This research was supported in part by NSF Grants CCR8809238 and CCR9157382 and in ...
The Dimensions of Individual Strings and Sequences
 INFORMATION AND COMPUTATION
, 2003
"... A constructive version of Hausdorff dimension is developed using constructive supergales, which are betting strategies that generalize the constructive supermartingales used in the theory of individual random sequences. This constructive dimension is used to assign every individual (infinite, binary ..."
Abstract

Cited by 93 (10 self)
 Add to MetaCart
A constructive version of Hausdorff dimension is developed using constructive supergales, which are betting strategies that generalize the constructive supermartingales used in the theory of individual random sequences. This constructive dimension is used to assign every individual (infinite, binary) sequence S a dimension, which is a real number dim(S) in the interval [0, 1]. Sequences that
The quantitative structure of exponential time
 Complexity theory retrospective II
, 1997
"... ABSTRACT Recent results on the internal, measuretheoretic structure of the exponential time complexity classes E and EXP are surveyed. The measure structure of these classes is seen to interact in informative ways with biimmunity, complexity cores, polynomialtime reductions, completeness, circuit ..."
Abstract

Cited by 90 (13 self)
 Add to MetaCart
ABSTRACT Recent results on the internal, measuretheoretic structure of the exponential time complexity classes E and EXP are surveyed. The measure structure of these classes is seen to interact in informative ways with biimmunity, complexity cores, polynomialtime reductions, completeness, circuitsize complexity, Kolmogorov complexity, natural proofs, pseudorandom generators, the density of hard languages, randomized complexity, and lowness. Possible implications for the structure of NP are also discussed. 1
Effective strong dimension in algorithmic information and computational complexity
 SIAM Journal on Computing
, 2004
"... The two most important notions of fractal dimension are Hausdorff dimension, developed by Hausdorff (1919), and packing dimension, developed independently by Tricot (1982) and Sullivan (1984). Both dimensions have the mathematical advantage of being defined from measures, and both have yielded exten ..."
Abstract

Cited by 79 (29 self)
 Add to MetaCart
The two most important notions of fractal dimension are Hausdorff dimension, developed by Hausdorff (1919), and packing dimension, developed independently by Tricot (1982) and Sullivan (1984). Both dimensions have the mathematical advantage of being defined from measures, and both have yielded extensive applications in fractal geometry and dynamical systems. Lutz (2000) has recently proven a simple characterization of Hausdorff dimension in terms of gales, which are betting strategies that generalize martingales. Imposing various computability and complexity constraints on these gales produces a spectrum of effective versions of Hausdorff dimension, including constructive, computable, polynomialspace, polynomialtime, and finitestate dimensions. Work by several investigators has already used these effective dimensions to shed significant new light on a variety of topics in theoretical computer science. In this paper we show that packing dimension can also be characterized in terms of gales. Moreover, even though the usual definition of packing dimension is considerably more complex than that of Hausdorff dimension, our gale characterization of packing dimension is an exact dual
Equivalence of Measures of Complexity Classes
"... The resourcebounded measures of complexity classes are shown to be robust with respect to certain changes in the underlying probability measure. Specifically, for any real number ffi ? 0, any uniformly polynomialtime computable sequence ~ fi = (fi 0 ; fi 1 ; fi 2 ; : : : ) of real numbers (biases ..."
Abstract

Cited by 70 (19 self)
 Add to MetaCart
The resourcebounded measures of complexity classes are shown to be robust with respect to certain changes in the underlying probability measure. Specifically, for any real number ffi ? 0, any uniformly polynomialtime computable sequence ~ fi = (fi 0 ; fi 1 ; fi 2 ; : : : ) of real numbers (biases) fi i 2 [ffi; 1 \Gamma ffi], and any complexity class C (such as P, NP, BPP, P/Poly, PH, PSPACE, etc.) that is closed under positive, polynomialtime, truthtable reductions with queries of at most linear length, it is shown that the following two conditions are equivalent. (1) C has pmeasure 0 (respectively, measure 0 in E, measure 0 in E 2 ) relative to the cointoss probability measure given by the sequence ~ fi. (2) C has pmeasure 0 (respectively, measure 0 in E, measure 0 in E 2 ) relative to the uniform probability measure. The proof introduces three techniques that may be useful in other contexts, namely, (i) the transformation of an efficient martingale for one probability measu...
Cook versus KarpLevin: Separating Completeness Notions If NP Is Not Small
 Theoretical Computer Science
, 1992
"... Under the hypothesis that NP does not have pmeasure 0 (roughly, that NP contains more than a negligible subset of exponential time), it is show n that there is a language that is P T complete ("Cook complete "), but not P m complete ("KarpLevin complete"), for NP. This conclusion, widely be ..."
Abstract

Cited by 56 (12 self)
 Add to MetaCart
Under the hypothesis that NP does not have pmeasure 0 (roughly, that NP contains more than a negligible subset of exponential time), it is show n that there is a language that is P T complete ("Cook complete "), but not P m complete ("KarpLevin complete"), for NP. This conclusion, widely believed to be true, is not known to follow from P 6= NP or other traditional complexitytheoretic hypotheses. Evidence is presented that "NP does not have pmeasure 0" is a reasonable hypothesis with many credible consequences. Additional such consequences proven here include the separation of many truthtable reducibilities in NP (e.g., k queries versus k+1 queries), the class separation E 6= NE, and the existence of NP search problems that are not reducible to the corresponding decision problems. This research was supported in part by National Science Foundation Grant CCR9157382, with matching funds from Rockwell International. 1 Introduction The NPcompleteness of decision problems has...
Discovering Neural Nets With Low Kolmogorov Complexity And High Generalization Capability
 Neural Networks
, 1997
"... Many neural net learning algorithms aim at finding "simple" nets to explain training data. The expectation is: the "simpler" the networks, the better the generalization on test data (! Occam's razor). Previous implementations, however, use measures for "simplicity" that lack the power, universali ..."
Abstract

Cited by 50 (31 self)
 Add to MetaCart
Many neural net learning algorithms aim at finding "simple" nets to explain training data. The expectation is: the "simpler" the networks, the better the generalization on test data (! Occam's razor). Previous implementations, however, use measures for "simplicity" that lack the power, universality and elegance of those based on Kolmogorov complexity and Solomonoff's algorithmic probability. Likewise, most previous approaches (especially those of the "Bayesian" kind) suffer from the problem of choosing appropriate priors. This paper addresses both issues. It first reviews some basic concepts of algorithmic complexity theory relevant to machine learning, and how the SolomonoffLevin distribution (or universal prior) deals with the prior problem. The universal prior leads to a probabilistic method for finding "algorithmically simple" problem solutions with high generalization capability. The method is based on Levin complexity (a timebounded generalization of Kolmogorov comple...
Randomness, relativization, and Turing degrees
 J. Symbolic Logic
, 2005
"... We compare various notions of algorithmic randomness. First we consider relativized randomness. A set is nrandom if it is MartinLof random relative to . We show that a set is 2random if and only if there is a constant c such that infinitely many initial segments x of the set are cincompre ..."
Abstract

Cited by 38 (16 self)
 Add to MetaCart
We compare various notions of algorithmic randomness. First we consider relativized randomness. A set is nrandom if it is MartinLof random relative to . We show that a set is 2random if and only if there is a constant c such that infinitely many initial segments x of the set are cincompressible: C(x) c. The `only if' direction was obtained independently by Joseph Miller. This characterization can be extended to the case of timebounded Ccomplexity.
System Identification, Approximation and Complexity
 International Journal of General Systems
, 1977
"... This paper is concerned with establishing broadlybased systemtheoretic foundations and practical techniques for the problem of system identification that are rigorous, intuitively clear and conceptually powerful. A general formulation is first given in which two order relations are postulated on a ..."
Abstract

Cited by 34 (23 self)
 Add to MetaCart
This paper is concerned with establishing broadlybased systemtheoretic foundations and practical techniques for the problem of system identification that are rigorous, intuitively clear and conceptually powerful. A general formulation is first given in which two order relations are postulated on a class of models: a constant one of complexity; and a variable one of approximation induced by an observed behaviour. An admissible model is such that any less complex model is a worse approximation. The general problem of identification is that of finding the admissible subspace of models induced by a given behaviour. It is proved under very general assumptions that, if deterministic models are required then nearly all behaviours require models of nearly maximum complexity. A general theory of approximation between models and behaviour is then developed based on subjective probability concepts and semantic information theory The role of structural constraints such as causality, locality, finite memory, etc., are then discussed as rules of the game. These concepts and results are applied to the specific problem or stochastic automaton, or grammar, inference. Computational results are given to demonstrate that the theory is complete and fully operational. Finally the formulation of identification proposed in this paper is analysed in terms of Klir’s epistemological hierarchy and both are discussed in terms of the rich philosophical literature on the acquisition of knowledge. 1
Prequential Probability: Principles and Properties
, 1997
"... this paper we first illustrate the above considerations for a variety of appealling criteria, and then, in an attempt to understand this behaviour, introduce a new gametheoretic framework for Probability Theory, the `prequential framework', which is particularly suited for the study of such problem ..."
Abstract

Cited by 33 (2 self)
 Add to MetaCart
this paper we first illustrate the above considerations for a variety of appealling criteria, and then, in an attempt to understand this behaviour, introduce a new gametheoretic framework for Probability Theory, the `prequential framework', which is particularly suited for the study of such problems.