. We investigate the distribution of nonuniform complexities in uniform complexity classes. We prove that almost every problem decidable in exponential space has essentially maximum circuit-size and space-bounded Kolmogorov complexity almost everywhere. (The circuit-size lower bound actually exceeds, and thereby strengthens, the Shannon 2 n n lower bound for almost every problem, with no computability constraint.) In exponential time complexity classes, we prove that the strongest relativizable lower bounds hold almost everywhere for almost all problems. Finally, we show that infinite pseudorandom sequences have high nonuniform complexity almost everywhere. The results are unified by a new, more powerful formulation of the underlying measure theory, based on uniform systems of density functions, and by the introduction of a new nonuniform complexity measure, the selective Kolmogorov complexity. This research was supported in part by NSF Grants CCR-8809238 and CCR-9157382 and in ...

ABSTRACT Recent results on the internal, measure-theoretic structure of the exponential time complexity classes E and EXP are surveyed. The measure structure of these classes is seen to interact in informative ways with bi-immunity, complexity cores, polynomial-time reductions, completeness, circuit-size complexity, Kolmogorov complexity, natural proofs, pseudorandom generators, the density of hard languages, randomized complexity, and lowness. Possible implications for the structure of NP are also discussed. 1

The two most important notions of fractal dimension are Hausdorff dimension, developed by Hausdorff (1919), and packing dimension, developed independently by Tricot (1982) and Sullivan (1984). Both dimensions have the mathematical advantage of being defined from measures, and both have yielded extensive applications in fractal geometry and dynamical systems. Lutz (2000) has recently proven a simple characterization of Hausdorff dimension in terms of gales, which are betting strategies that generalize martingales. Imposing various computability and complexity constraints on these gales produces a spectrum of effective versions of Hausdorff dimension, including constructive, computable, polynomial-space, polynomial-time, and finite-state dimensions. Work by several investigators has already used these effective dimensions to shed significant new light on a variety of topics in theoretical computer science. In this paper we show that packing dimension can also be characterized in terms of gales. Moreover, even though the usual definition of packing dimension is considerably more complex than that of Hausdorff dimension, our gale characterization of packing dimension is an exact dual

The resource-bounded measures of complexity classes are shown to be robust with respect to certain changes in the underlying probability measure. Specifically, for any real number ffi ? 0, any uniformly polynomial-time computable sequence ~ fi = (fi 0 ; fi 1 ; fi 2 ; : : : ) of real numbers (biases) fi i 2 [ffi; 1 \Gamma ffi], and any complexity class C (such as P, NP, BPP, P/Poly, PH, PSPACE, etc.) that is closed under positive, polynomial-time, truth-table reductions with queries of at most linear length, it is shown that the following two conditions are equivalent. (1) C has p-measure 0 (respectively, measure 0 in E, measure 0 in E 2 ) relative to the coin-toss probability measure given by the sequence ~ fi. (2) C has p-measure 0 (respectively, measure 0 in E, measure 0 in E 2 ) relative to the uniform probability measure. The proof introduces three techniques that may be useful in other contexts, namely, (i) the transformation of an efficient martingale for one probability measu...

Under the hypothesis that NP does not have p-measure 0 (roughly, that NP contains more than a negligible subset of exponential time), it is show n that there is a language that is P T -complete ("Cook complete "), but not P m -complete ("Karp-Levin complete"), for NP. This conclusion, widely believed to be true, is not known to follow from P 6= NP or other traditional complexity-theoretic hypotheses. Evidence is presented that "NP does not have p-measure 0" is a reasonable hypothesis with many credible consequences. Additional such consequences proven here include the separation of many truthtable reducibilities in NP (e.g., k queries versus k+1 queries), the class separation E 6= NE, and the existence of NP search problems that are not reducible to the corresponding decision problems. This research was supported in part by National Science Foundation Grant CCR9157382, with matching funds from Rockwell International. 1 Introduction The NP-completeness of decision problems has...

Many neural net learning algorithms aim at finding "simple" nets to explain training data. The expectation is: the "simpler" the networks, the better the generalization on test data (! Occam's razor). Previous implementations, however, use measures for "simplicity" that lack the power, universality and elegance of those based on Kolmogorov complexity and Solomonoff's algorithmic probability. Likewise, most previous approaches (especially those of the "Bayesian" kind) suffer from the problem of choosing appropriate priors. This paper addresses both issues. It first reviews some basic concepts of algorithmic complexity theory relevant to machine learning, and how the Solomonoff-Levin distribution (or universal prior) deals with the prior problem. The universal prior leads to a probabilistic method for finding "algorithmically simple" problem solutions with high generalization capability. The method is based on Levin complexity (a time-bounded generalization of Kolmogorov comple...

We prove that if strong pseudorandom number generators exist, then the class of languages that have polynomialsized circuits (P/poly) is not measurable within exponential time, in terms of the resource-bounded measure theory of Lutz. We prove our result by showing that if P/poly has measure zero in exponential time, then there is a natural proof against P/poly, in the terminology of Razborov and Rudich [25]. We also provide a partial converse of this result.

We compare various notions of algorithmic randomness. First we consider relativized randomness. A set is n-random if it is Martin-Lof random relative to . We show that a set is 2-random if and only if there is a constant c such that infinitely many initial segments x of the set are c-incompressible: C(x) c. The `only if' direction was obtained independently by Joseph Miller. This characterization can be extended to the case of time-bounded C-complexity.

this paper we first illustrate the above considerations for a variety of appealling criteria, and then, in an attempt to understand this behaviour, introduce a new game-theoretic framework for Probability Theory, the `prequential framework', which is particularly suited for the study of such problems.

Resource-bounded dimension is a complexity-theoretic extension of classical Hausdorff dimension introduced by Lutz (2000) in order to investigate the fractal structure of sets that have resource-bounded measure 0. For example, while it has long been known that the Boolean circuit-size complexity class SIZE � α 2n n has measure 0 in ESPACE for all 0 ≤ α ≤ 1, we now know that SIZE � α 2n n has dimension α in ESPACE for all 0 ≤ α ≤ 1. The present paper furthers this program by developing a natural hierarchy of “rescaled” resource-bounded dimensions. For each integer i and each set X of decision problems, we define the ith-order dimension of X in suitable complexity classes. The 0th-order dimension is precisely the dimension of Hausdorff (1919) and Lutz (2000). Higher and lower orders are useful for various sets X. For example, we prove the following for 0 ≤ α ≤ 1 and any polynomial q(n) ≥ n2. 1. The class SIZE(2 αn) and the time- and space-bounded Kolmogorov complexity classes KT q (2 αn) and KS q (2 αn) have 1 st-order dimension α in ESPACE. 2. The classes SIZE(2nα), KT q (2nα), and KS q (2nα) have 2nd-order dimension α in ESPACE.