Results 1  10
of
39
Fast Cryptographic Primitives and CircularSecure Encryption Based on Hard Learning Problems
"... Abstract. The wellstudied task of learning a linear function with errors is a seemingly hard problem and the basis for several cryptographic schemes. Here we demonstrate additional applications that enjoy strong security properties and a high level of efficiency. Namely, we construct: 1. Publickey ..."
Abstract

Cited by 37 (10 self)
 Add to MetaCart
Abstract. The wellstudied task of learning a linear function with errors is a seemingly hard problem and the basis for several cryptographic schemes. Here we demonstrate additional applications that enjoy strong security properties and a high level of efficiency. Namely, we construct: 1. Publickey and symmetrickey cryptosystems that provide security for keydependent messages and enjoy circular security. Our schemes are highly efficient: in both cases the ciphertext is only a constant factor larger than the plaintext, and the cost of encryption and decryption is only n · polylog(n) bit operations per message symbol in the publickey case, and polylog(n) bit operations in the symmetric case. 2. Two efficient pseudorandom objects: a “weak randomized pseudorandom function ” — a relaxation of standard PRF — that can be computed obliviously via a simple protocol, and a lengthdoubling pseudorandom generator that can be computed by a circuit of n ·
Cryptographic hardness for learning intersections of halfspaces
 J. Comput. Syst. Sci
"... We give the first representationindependent hardness results for PAC learning intersections of halfspaces, a central concept class in computational learning theory. Our hardness results are derived from two publickey cryptosystems due to Regev, which are based on the worstcase hardness of wellstu ..."
Abstract

Cited by 33 (13 self)
 Add to MetaCart
We give the first representationindependent hardness results for PAC learning intersections of halfspaces, a central concept class in computational learning theory. Our hardness results are derived from two publickey cryptosystems due to Regev, which are based on the worstcase hardness of wellstudied lattice problems. Specifically, we prove that a polynomialtime algorithm for PAC learning intersections of n ε halfspaces (for a constant ε> 0) in n dimensions would yield a polynomialtime solution to Õ(n 1.5)uSVP (unique shortest vector problem). We also prove that PAC learning intersections of n ε lowweight halfspaces would yield a polynomialtime quantum solution to Õ(n 1.5)SVP and Õ(n 1.5)SIVP (shortest vector problem and shortest independent vector problem, respectively). Our approach also yields the first representationindependent hardness results for learning polynomialsize depth2 neural networks and polynomialsize depth3 arithmetic circuits. Key words: Cryptographic hardness results, intersections of halfspaces, computational learning theory, latticebased cryptography 1
Hardness of learning halfspaces with noise
 In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science
, 2006
"... Learning an unknown halfspace (also called a perceptron) from labeled examples is one of the classic problems in machine learning. In the noisefree case, when a halfspace consistent with all the training examples exists, the problem can be solved in polynomial time using linear programming. However ..."
Abstract

Cited by 33 (3 self)
 Add to MetaCart
Learning an unknown halfspace (also called a perceptron) from labeled examples is one of the classic problems in machine learning. In the noisefree case, when a halfspace consistent with all the training examples exists, the problem can be solved in polynomial time using linear programming. However, under the promise that a halfspace consistent with a fraction (1 − ε) of the examples exists (for some small constant ε> 0), it was not known how to efficiently find a halfspace that is correct on even 51 % of the examples. Nor was a hardness result that ruled out getting agreement on more than 99.9 % of the examples known. In this work, we close this gap in our understanding, and prove that even a tiny amount of worstcase noise makes the problem of learning halfspaces intractable in a strong sense. Specifically, for arbitrary ε, δ> 0, we prove that given a set of exampleslabel pairs from the hypercube a fraction (1 − ε) of which can be explained by a halfspace, it is NPhard to find a halfspace that correctly labels a fraction (1/2 + δ) of the examples. The hardness result is tight since it is trivial to get agreement on 1/2 the examples. In learning theory parlance, we prove that weak proper agnostic learning of halfspaces is hard. This settles a question that was raised by Blum et al. in their work on learning halfspaces in the presence of random classification noise [10], and in some more recent works as well. Along the way, we also obtain a strong hardness result for another basic computational problem: solving a linear system over the rationals. 1
The signrank of AC^0
 IN PROC. OF THE 49TH SYMPOSIUM ON FOUNDATIONS OF COMPUTER SCIENCE (FOCS
, 2008
"... The signrank of a matrix A = [Ai j] with ±1 entries is the least rank of a real matrix B = [Bi j] with Ai j Bi j> 0 for all i, j. We obtain the first exponential lower bound on the signrank of a function in AC 0. Namely, let f (x, y) = �m �m2 i=1 j=1 (xi j ∧ yi j). We show that the matrix [ f (x, ..."
Abstract

Cited by 18 (9 self)
 Add to MetaCart
The signrank of a matrix A = [Ai j] with ±1 entries is the least rank of a real matrix B = [Bi j] with Ai j Bi j> 0 for all i, j. We obtain the first exponential lower bound on the signrank of a function in AC 0. Namely, let f (x, y) = �m �m2 i=1 j=1 (xi j ∧ yi j). We show that the matrix [ f (x, y)]x,y has signrank 2�(m). This in particular implies that �cc 2 � ⊆ UPPcc, which solves a longstanding open problem posed by Babai, Frankl, and Simon (1986). Our result additionally implies a lower bound in learning theory. Specifically, let φ1,..., φr: {0, 1} n → R be functions such that every DNF formula f: {0, 1} n → {−1, +1} of polynomial size has the representation f ≡ sign(a1φ1 + · · · + ar φr) for some reals a1,..., ar. We prove that then r � 2�(n1/3) , which essentially matches an upper bound of 2Õ(n1/3) due to Klivans and Servedio (2001). Finally, our work yields the first exponential lower bound on the size of thresholdofmajority circuits computing a function in AC 0. This substantially generalizes and strengthens the results of Krause and Pudlák (1997).
The unboundederror communication complexity of symmetric functions
 In Proc. of the 49th Symposium on Foundations of Computer Science (FOCS
, 2008
"... We prove an essentially tight lower bound on the unboundederror communication complexity of every symmetric function, i.e., f (x, y) = D(x ∧ y), where D: {0, 1,..., n} → {0, 1} is a given predicate and x, y range over {0, 1} n. Specifically, we show that the communication complexity of f is betw ..."
Abstract

Cited by 16 (9 self)
 Add to MetaCart
We prove an essentially tight lower bound on the unboundederror communication complexity of every symmetric function, i.e., f (x, y) = D(x ∧ y), where D: {0, 1,..., n} → {0, 1} is a given predicate and x, y range over {0, 1} n. Specifically, we show that the communication complexity of f is between �(k / log5 n) and �(k log n), where k is the number of value changes of D in {0, 1,..., n}. The unboundederror model is the most powerful of the basic models of communication (both classical and quantum), and proving lower bounds in it is a considerable challenge. The only previous nontrivial lower bounds for explicit functions in this model appear in the groundbreaking work of Forster (2001) and its extensions. Our proof is built around two novel ideas. First, we show that a given predicate D gives rise to a rapidly mixing random walk on Zn 2, which allows us to reduce the problem to communication lower bounds for “typical” predicates. Second, we use Paturi’s approximation lower bounds (1992), suitably generalized here to clusters of real nodes in [0, n] and interpreted in their dual form, to prove that a typical predicate behaves analogous to PARITY with respect to a smooth distribution on the inputs.
Unconditional lower bounds for learning intersections of halfspaces
 Machine Learning
, 2007
"... We prove new lower bounds for learning intersections of halfspaces, one of the most important concept classes in computational learning theory. Our main result is that any statisticalquery algorithm for learning the intersection of √ n halfspaces in n dimensions must make 2 Ω( √ n) queries. This is ..."
Abstract

Cited by 16 (12 self)
 Add to MetaCart
We prove new lower bounds for learning intersections of halfspaces, one of the most important concept classes in computational learning theory. Our main result is that any statisticalquery algorithm for learning the intersection of √ n halfspaces in n dimensions must make 2 Ω( √ n) queries. This is the first nontrivial lower bound on the statistical query dimension for this concept class (the previous best lower bound was n Ω(logn)). Our lower bound holds even for intersections of lowweight halfspaces. In the latter case, it is nearly tight. We also show that the intersection of two majorities (lowweight halfspaces) cannot be computed by a polynomial threshold function (PTF) with fewer than n Ω(logn/loglogn) monomials. This is the first superpolynomial lower bound on the PTF length of this concept class, and is nearly optimal. For intersections of k = ω(logn) lowweight halfspaces, we improve our lower bound to min{2 Ω( √ n),n Ω(k/logk)}, which too is nearly optimal. As a consequence, intersections of even two halfspaces are not computable by polynomialweight PTFs, the most expressive class of functions known to be efficiently learnable via Jackson’s Harmonic Sieve algorithm. Finally, we report our progress on the weak learnability of intersections of halfspaces under the uniform distribution. 1
Differentially Private Data Release through Multidimensional Partitioning
"... Abstract. Differential privacy is a strong notion for protecting individual privacy in privacy preserving data analysis or publishing. In this paper, we study the problem of differentially private histogram release based on an interactive differential privacy interface. We propose two multidimension ..."
Abstract

Cited by 16 (8 self)
 Add to MetaCart
Abstract. Differential privacy is a strong notion for protecting individual privacy in privacy preserving data analysis or publishing. In this paper, we study the problem of differentially private histogram release based on an interactive differential privacy interface. We propose two multidimensional partitioning strategies including a baseline cellbased partitioning and an innovative kdtree based partitioning. In addition to providing formal proofs for differential privacy and usefulness guarantees for linear distributive queries, we also present a set of experimental results and demonstrate the feasibility and performance of our method. 1
New algorithms for learning in presence of errors
 ICALP
"... We give new algorithms for a variety of randomlygenerated instances of computational problems using a linearization technique that reduces to solving a system of linear equations. These algorithms are derived in the context of learning with structured noise, a notion introduced in this paper. This ..."
Abstract

Cited by 14 (0 self)
 Add to MetaCart
We give new algorithms for a variety of randomlygenerated instances of computational problems using a linearization technique that reduces to solving a system of linear equations. These algorithms are derived in the context of learning with structured noise, a notion introduced in this paper. This notion is best illustrated with the learning parities with noise (LPN) problem —wellstudied in learning theory and cryptography. In the standard version, we have access to an oracle that, each time we press a button, returns a random vector a ∈ GF(2) n together with a bit b ∈ GF(2) that was computed as a · u + η, where u ∈ GF(2) n is a secret vector, and η ∈ GF(2) is a noise bit that is 1 with some probability p. Say p = 1/3. The goal is to recover u. This task is conjectured to be intractable. In the structured noise setting we introduce a slight (?) variation of the model: upon pressing a button, we receive (say) 10 random vectors a1, a2,..., a10 ∈ GF(2) n, and corresponding bits b1, b2,..., b10, of which at most 3 are noisy. The oracle may arbitrarily decide which of the 10 bits to make noisy. We exhibit a polynomialtime algorithm to recover the secret vector u given such an oracle. We think this structured noise model may be of independent interest in machine learning. We discuss generalizations of our result, including learning with more general noise patterns. We also give the first nontrivial algorithms for two problems, which we show fit in our structured noise framework. We give a slightly subexponential algorithm for the wellknown learning with errors (LWE) problem over GF(q) introduced by Regev for cryptographic uses. Our algorithm works for the case when the gaussian noise is small; which was an open problem. We also give polynomialtime algorithms for learning the MAJORITY OF PARITIES function of Applebaum et al. for certain parameter values. This function is a special case of Goldreich’s pseudorandom generator. Research supported by NSF Grants CCF0832797, 0830673, and 0528414
Agnostic Learning of Monomials by Halfspaces is Hard
"... Abstract — We prove the following strong hardness result for learning: Given a distribution on labeled examples from the hypercube such that there exists a monomial (or conjunction) consistent with (1 − ϵ)fraction of the examples, it is NPhard to find a halfspace that is correct on ( 1 +ϵ)fractio ..."
Abstract

Cited by 13 (7 self)
 Add to MetaCart
Abstract — We prove the following strong hardness result for learning: Given a distribution on labeled examples from the hypercube such that there exists a monomial (or conjunction) consistent with (1 − ϵ)fraction of the examples, it is NPhard to find a halfspace that is correct on ( 1 +ϵ)fraction of the examples, 2 for arbitrary constant ϵ> 0. In learning theory terms, weak agnostic learning of monomials by halfspaces is NPhard. This hardness result bridges between and subsumes two previous results which showed similar hardness results for the proper learning of monomials and halfspaces. As immediate corollaries of our result, we give the first optimal hardness results for weak agnostic learning of decision lists and majorities. Our techniques are quite different from previous hardness proofs for learning. We use an invariance principle and sparse approximation of halfspaces from recent work on fooling halfspaces to give a new natural list decoding of a halfspace in the context of dictatorship tests/label cover reductions. In addition, unlike previous invariance principle based proofs which are only known to give Unique Games hardness, we give a reduction from a smooth version of Label Cover that is known to be NPhard.
Improved Guarantees for Learning via Similarity Functions
"... We continue the investigation of natural conditions for a similarity function to allow learning, without requiring the similarity function to be a valid kernel, or referring to an implicit highdimensional space. We provide a new notion of a “good similarity function ” that builds upon the previous ..."
Abstract

Cited by 10 (2 self)
 Add to MetaCart
We continue the investigation of natural conditions for a similarity function to allow learning, without requiring the similarity function to be a valid kernel, or referring to an implicit highdimensional space. We provide a new notion of a “good similarity function ” that builds upon the previous definition of Balcan and Blum (2006) but improves on it in two important ways. First, as with the previous definition, any largemargin kernel is also a good similarity function in our sense, but the translation now results in a much milder increase in the labeled sample complexity. Second, we prove that for distributionspecific PAC learning, our new notion is strictly more powerful than the traditional notion of a largemargin kernel. In particular, we show that for any hypothesis class C there exists a similarity function under our definition allowing learning with O(log C) labeled examples. However, in a lower bound which may be of independent interest, we show that for any class C of pairwise uncorrelated functions, there is no kernel with margin γ ≥ 8 / √ C  for all f ∈ C, even if one allows average hingeloss as large as 0.5. Thus, the sample complexity for learning such classes with SVMs is Ω(C). This extends work of BenDavid et al. (2003) and Forster and Simon (2006) who give hardness results with comparable margin bounds, but at much lower error rates. Our new notion of similarity relies upon L1 regularized learning, and our separation result is related to a separation result between what is learnable with L1 vs. L2 regularization. 1