In this paper we prove the intractability of learning several classes of Boolean functions in the distribution-free model (also called the Probably Approximately Correct or PAC model) of learning from examples. These results are representation independent, in that they hold regardless of the syntactic form in which the learner chooses to represent its hypotheses. Our methods reduce the problems of cracking a number of well-known public-key cryptosystems to the learning problems. We prove that a polynomial-time learning algorithm for Boolean formulae, deterministic finite automata or constant-depth threshold circuits would have dramatic consequences for cryptography and number theory: in particular, such an algorithm could be used to break the RSA cryptosystem, factor Blum integers (composite numbers equivalent to 3 modulo 4), and detect quadratic residues. The results hold even if the learning algorithm is only required to obtain a slight advantage in prediction over random guessing. The techniques used demonstrate an interesting duality between learning and cryptography. We also apply our results to obtain strong intractability results for approximating a generalization of graph coloring.

We show that any language recognized by an NC ’ circuit (fan-in 2, depth O(log n)) can be recognized by a width-5 polynomial-size branching program. As any bounded-width polynomial-size branching program can be simulated by an NC ’ circuit, we have that the class of languages recognized by such programs is exactly nonuniform NC’. Further, following

In order to study circuit complexity classes within NC¹ in a uniform setting, we need a uniformity condition which is more restrictive than those in common use. Two such conditions, stricter than NC¹ uniformity [Ru81,Co85], have appeared in recent research: Immerman's families of circuits defined by first-order formulas [Im87a,Im87b] and a uniformity corresponding to Buss' deterministic log-time reductions [Bu87]. We show that these two notions are equivalent, leading to a natural notion of uniformity for low-level circuit complexity classes. We show that recent results on the structure of NC¹ [Ba89] still hold true in this very uniform setting. Finally, we investigate a parallel notion of uniformity, still more restrictive, based on the regular languages. Here we give characterizations of subclasses of the regular languages based on their logical expressibility, extending recent work of Straubing, Th'erien, and Thomas [STT88]. A preliminary version of this work appeared as [BIS88].

Abstract. Weinvestigate the power of threshold circuits of small depth. In particular, we give functions that require exponential size unweighted threshold circuits of depth 3 when we restrict the bottom fanin. We also prove that there are monotone functions fk that can be computed in depth k and linear size ^ � _-circuits but require exponential size to compute by a depth k; 1 monotone weighted threshold circuit. Key words. Circuit complexity, monotone circuits, threshold circuits, lower bounds Subject classi cations. 68Q15, 68Q99 1.

We pursue a particular approach to analog computation, based on dynamical systems of the type used in neural networks research. Our systems have a fixed structure, invariant in time, corresponding to an unchanging number of "neurons". If allowed exponential time for computation, they turn out to have unbounded power. However, under polynomial-time constraints there are limits on their capabilities, though being more powerful than Turing Machines. (A similar but more restricted model was shown to be polynomial-time equivalent to classical digital computation in the previous work [20].) Moreover, there is a precise correspondence between nets and standard non-uniform circuits with equivalent resources, and as a consequence one has lower bound constraints on what they can compute. This relationship is perhaps surprising since our analog devices do not change in any manner with input size. We note that these networks are not likely to solve polynomially NP-hard problems, as the equality ...

. It is shown that high order feedforward neural nets of constant depth with piecewise polynomial activation functions and arbitrary real weights can be simulated for boolean inputs and outputs by neural nets of a somewhat larger size and depth with heaviside gates and weights from f\Gamma1; 0; 1g. This provides the first known upper bound for the computational power of the former type of neural nets. It is also shown that in the case of first order nets with piecewise linear activation functions one can replace arbitrary real weights by rational numbers with polynomially many bits, without changing the boolean function that is computed by the neural net. In order to prove these results we introduce two new methods for reducing nonlinear problems about weights in multi-layer neural nets to linear problems for a transformed set of parameters. These transformed parameters can be interpreted as weights in a somewhat larger neural net. As another application of our new proof technique we s...

The representation of functions as low-degree polynomials over various rings has provided many insights in the theory of small-depth circuits. We survey some of the closure properties, upper bounds, and lower bounds obtained via this approach. 1. Introduction There is a long history of using polynomials in order to prove complexity bounds. Minsky and Papert [39] used polynomials to prove early lower bounds on the order of perceptrons. Razborov [46] and Smolensky [49] used them to prove lower bounds on the size of AND-OR circuits. Other lower bounds via polynomials are due to [50, 4, 10, 51, 9, 55]. Paturi and Saks [44] discovered that rational functions could be used for lower bounds on the size of threshold circuits. Toda [53] used polynomials to prove upper bounds on the power of the polynomial hierarchy. This led to a series of upper bounds on the power of the polynomial hierarchy [54, 52], AC 0 [2, 3, 52, 19], and ACC [58, 20, 30, 37], and related classes [21, 42]. Beigel and Gi...

Counting classes consist of languages defined in terms of the number of accepting computations of nondeterministic polynomial-time Turing machines. Well known examples of counting classes are NP, co-NP, \PhiP, and PP. Every counting class is a subset of P #P[1] , the class of languages computable in polynomial time using a single call to an oracle capable of determining the number of accepting paths of an NP machine. Using closure properties of #P, we systematically develop a complexity theory for counting classes defined in terms of thresholds and moduli. An unexpected result is that MOD k iP = MOD k P for prime k. Finally, we improve a result of Cai and Hemachandra by showing that recognizing languages in the class Few is as easy as distinguishing uniquely satisfiable formulas from unsatisfiable formulas (or detecting unique solutions, as in [28]). 1. Introduction Valiant [27] defined the class #P of functions whose values equal the number of accepting paths of polynomial-time bo...

Abstract. Optimal Q(logn/log logn) lower bounds on the time for CRCW PRAMS with polynomially bounded numbers of processors or memory cells to compute parity and a number of related problems are proven. A strict time hierarchy of explicit Boolean functions of n bits on such machines that holds up to O(logn/loglogn) time is also exhibited. That is, for every time bound T within this range a function is exhibited that can be easily computed using polynomial resources in time T but requires more than polynomial resources to be computed in time T- 1. Finally, it is shown that almost all Boolean functions of n bits require logn- loglogn + fi ( 1) time when the number of processors is at most polynomial in n. The bounds do not place restrictions on the uniformity of the algorithms nor on the instruction sets of the machines.

The interpolation method has been one of the main tools for proving lower bounds for propositional proof systems. Loosely speaking, if one can prove that a particular proof system has the feasible interpolation property, then a generic reduction can (usually) be applied to prove lower bounds for the proof system, sometimes assuming a (usually modest) complexity-theoretic assumption. In this paper, we show that this method cannot be used to obtain lower bounds for Frege systems, or even for TC 0 -Frege systems. More specifically, we show that unless factoring (of Blum integers) is feasible, neither Frege nor TC 0 -Frege has the feasible interpolation property. In order to carry out our argument, we show how to carry out proofs of many elementary axioms/theorems of arithmetic in polynomial size TC 0 -Frege. As a corollary, we obtain that TC 0 -Frege as well as any proof system that polynomially simulates it, is not automatizable (under the assumption that factoring of Blum integ...