Results 1  10
of
48
Fast Cryptographic Primitives and CircularSecure Encryption Based on Hard Learning Problems
"... Abstract. The wellstudied task of learning a linear function with errors is a seemingly hard problem and the basis for several cryptographic schemes. Here we demonstrate additional applications that enjoy strong security properties and a high level of efficiency. Namely, we construct: 1. Publickey ..."
Abstract

Cited by 38 (11 self)
 Add to MetaCart
Abstract. The wellstudied task of learning a linear function with errors is a seemingly hard problem and the basis for several cryptographic schemes. Here we demonstrate additional applications that enjoy strong security properties and a high level of efficiency. Namely, we construct: 1. Publickey and symmetrickey cryptosystems that provide security for keydependent messages and enjoy circular security. Our schemes are highly efficient: in both cases the ciphertext is only a constant factor larger than the plaintext, and the cost of encryption and decryption is only n · polylog(n) bit operations per message symbol in the publickey case, and polylog(n) bit operations in the symmetric case. 2. Two efficient pseudorandom objects: a “weak randomized pseudorandom function ” — a relaxation of standard PRF — that can be computed obliviously via a simple protocol, and a lengthdoubling pseudorandom generator that can be computed by a circuit of n ·
Hardness of learning halfspaces with noise
 In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science
, 2006
"... Learning an unknown halfspace (also called a perceptron) from labeled examples is one of the classic problems in machine learning. In the noisefree case, when a halfspace consistent with all the training examples exists, the problem can be solved in polynomial time using linear programming. However ..."
Abstract

Cited by 30 (3 self)
 Add to MetaCart
Learning an unknown halfspace (also called a perceptron) from labeled examples is one of the classic problems in machine learning. In the noisefree case, when a halfspace consistent with all the training examples exists, the problem can be solved in polynomial time using linear programming. However, under the promise that a halfspace consistent with a fraction (1 − ε) of the examples exists (for some small constant ε> 0), it was not known how to efficiently find a halfspace that is correct on even 51 % of the examples. Nor was a hardness result that ruled out getting agreement on more than 99.9 % of the examples known. In this work, we close this gap in our understanding, and prove that even a tiny amount of worstcase noise makes the problem of learning halfspaces intractable in a strong sense. Specifically, for arbitrary ε, δ> 0, we prove that given a set of exampleslabel pairs from the hypercube a fraction (1 − ε) of which can be explained by a halfspace, it is NPhard to find a halfspace that correctly labels a fraction (1/2 + δ) of the examples. The hardness result is tight since it is trivial to get agreement on 1/2 the examples. In learning theory parlance, we prove that weak proper agnostic learning of halfspaces is hard. This settles a question that was raised by Blum et al. in their work on learning halfspaces in the presence of random classification noise [10], and in some more recent works as well. Along the way, we also obtain a strong hardness result for another basic computational problem: solving a linear system over the rationals. 1
The signrank of AC^0
 IN PROC. OF THE 49TH SYMPOSIUM ON FOUNDATIONS OF COMPUTER SCIENCE (FOCS
, 2008
"... The signrank of a matrix A = [Ai j] with ±1 entries is the least rank of a real matrix B = [Bi j] with Ai j Bi j> 0 for all i, j. We obtain the first exponential lower bound on the signrank of a function in AC 0. Namely, let f (x, y) = �m �m2 i=1 j=1 (xi j ∧ yi j). We show that the matrix [ f ..."
Abstract

Cited by 20 (10 self)
 Add to MetaCart
The signrank of a matrix A = [Ai j] with ±1 entries is the least rank of a real matrix B = [Bi j] with Ai j Bi j> 0 for all i, j. We obtain the first exponential lower bound on the signrank of a function in AC 0. Namely, let f (x, y) = �m �m2 i=1 j=1 (xi j ∧ yi j). We show that the matrix [ f (x, y)]x,y has signrank 2�(m). This in particular implies that �cc 2 � ⊆ UPPcc, which solves a longstanding open problem posed by Babai, Frankl, and Simon (1986). Our result additionally implies a lower bound in learning theory. Specifically, let φ1,..., φr: {0, 1} n → R be functions such that every DNF formula f: {0, 1} n → {−1, +1} of polynomial size has the representation f ≡ sign(a1φ1 + · · · + ar φr) for some reals a1,..., ar. We prove that then r � 2�(n1/3) , which essentially matches an upper bound of 2Õ(n1/3) due to Klivans and Servedio (2001). Finally, our work yields the first exponential lower bound on the size of thresholdofmajority circuits computing a function in AC 0. This substantially generalizes and strengthens the results of Krause and Pudlák (1997).
New algorithms for learning in presence of errors
 ICALP
"... We give new algorithms for a variety of randomlygenerated instances of computational problems using a linearization technique that reduces to solving a system of linear equations. These algorithms are derived in the context of learning with structured noise, a notion introduced in this paper. This ..."
Abstract

Cited by 17 (0 self)
 Add to MetaCart
We give new algorithms for a variety of randomlygenerated instances of computational problems using a linearization technique that reduces to solving a system of linear equations. These algorithms are derived in the context of learning with structured noise, a notion introduced in this paper. This notion is best illustrated with the learning parities with noise (LPN) problem —wellstudied in learning theory and cryptography. In the standard version, we have access to an oracle that, each time we press a button, returns a random vector a ∈ GF(2) n together with a bit b ∈ GF(2) that was computed as a · u + η, where u ∈ GF(2) n is a secret vector, and η ∈ GF(2) is a noise bit that is 1 with some probability p. Say p = 1/3. The goal is to recover u. This task is conjectured to be intractable. In the structured noise setting we introduce a slight (?) variation of the model: upon pressing a button, we receive (say) 10 random vectors a1, a2,..., a10 ∈ GF(2) n, and corresponding bits b1, b2,..., b10, of which at most 3 are noisy. The oracle may arbitrarily decide which of the 10 bits to make noisy. We exhibit a polynomialtime algorithm to recover the secret vector u given such an oracle. We think this structured noise model may be of independent interest in machine learning. We discuss generalizations of our result, including learning with more general noise patterns. We also give the first nontrivial algorithms for two problems, which we show fit in our structured noise framework. We give a slightly subexponential algorithm for the wellknown learning with errors (LWE) problem over GF(q) introduced by Regev for cryptographic uses. Our algorithm works for the case when the gaussian noise is small; which was an open problem. We also give polynomialtime algorithms for learning the MAJORITY OF PARITIES function of Applebaum et al. for certain parameter values. This function is a special case of Goldreich’s pseudorandom generator. Research supported by NSF Grants CCF0832797, 0830673, and 0528414
Differentially Private Data Release through Multidimensional Partitioning
"... Abstract. Differential privacy is a strong notion for protecting individual privacy in privacy preserving data analysis or publishing. In this paper, we study the problem of differentially private histogram release based on an interactive differential privacy interface. We propose two multidimension ..."
Abstract

Cited by 17 (8 self)
 Add to MetaCart
Abstract. Differential privacy is a strong notion for protecting individual privacy in privacy preserving data analysis or publishing. In this paper, we study the problem of differentially private histogram release based on an interactive differential privacy interface. We propose two multidimensional partitioning strategies including a baseline cellbased partitioning and an innovative kdtree based partitioning. In addition to providing formal proofs for differential privacy and usefulness guarantees for linear distributive queries, we also present a set of experimental results and demonstrate the feasibility and performance of our method. 1
The unboundederror communication complexity of symmetric functions
 In Proc. of the 49th Symposium on Foundations of Computer Science (FOCS
, 2008
"... We prove an essentially tight lower bound on the unboundederror communication complexity of every symmetric function, i.e., f (x, y) = D(x ∧ y), where D: {0, 1,..., n} → {0, 1} is a given predicate and x, y range over {0, 1} n. Specifically, we show that the communication complexity of f is betw ..."
Abstract

Cited by 16 (9 self)
 Add to MetaCart
We prove an essentially tight lower bound on the unboundederror communication complexity of every symmetric function, i.e., f (x, y) = D(x ∧ y), where D: {0, 1,..., n} → {0, 1} is a given predicate and x, y range over {0, 1} n. Specifically, we show that the communication complexity of f is between �(k / log5 n) and �(k log n), where k is the number of value changes of D in {0, 1,..., n}. The unboundederror model is the most powerful of the basic models of communication (both classical and quantum), and proving lower bounds in it is a considerable challenge. The only previous nontrivial lower bounds for explicit functions in this model appear in the groundbreaking work of Forster (2001) and its extensions. Our proof is built around two novel ideas. First, we show that a given predicate D gives rise to a rapidly mixing random walk on Zn 2, which allows us to reduce the problem to communication lower bounds for “typical” predicates. Second, we use Paturi’s approximation lower bounds (1992), suitably generalized here to clusters of real nodes in [0, n] and interpreted in their dual form, to prove that a typical predicate behaves analogous to PARITY with respect to a smooth distribution on the inputs.
Unconditional lower bounds for learning intersections of halfspaces
 Machine Learning
, 2007
"... We prove new lower bounds for learning intersections of halfspaces, one of the most important concept classes in computational learning theory. Our main result is that any statisticalquery algorithm for learning the intersection of √ n halfspaces in n dimensions must make 2 Ω( √ n) queries. This is ..."
Abstract

Cited by 16 (12 self)
 Add to MetaCart
We prove new lower bounds for learning intersections of halfspaces, one of the most important concept classes in computational learning theory. Our main result is that any statisticalquery algorithm for learning the intersection of √ n halfspaces in n dimensions must make 2 Ω( √ n) queries. This is the first nontrivial lower bound on the statistical query dimension for this concept class (the previous best lower bound was n Ω(logn)). Our lower bound holds even for intersections of lowweight halfspaces. In the latter case, it is nearly tight. We also show that the intersection of two majorities (lowweight halfspaces) cannot be computed by a polynomial threshold function (PTF) with fewer than n Ω(logn/loglogn) monomials. This is the first superpolynomial lower bound on the PTF length of this concept class, and is nearly optimal. For intersections of k = ω(logn) lowweight halfspaces, we improve our lower bound to min{2 Ω( √ n),n Ω(k/logk)}, which too is nearly optimal. As a consequence, intersections of even two halfspaces are not computable by polynomialweight PTFs, the most expressive class of functions known to be efficiently learnable via Jackson’s Harmonic Sieve algorithm. Finally, we report our progress on the weak learnability of intersections of halfspaces under the uniform distribution. 1
Agnostic Learning of Monomials by Halfspaces is Hard
"... Abstract — We prove the following strong hardness result for learning: Given a distribution on labeled examples from the hypercube such that there exists a monomial (or conjunction) consistent with (1 − ϵ)fraction of the examples, it is NPhard to find a halfspace that is correct on ( 1 +ϵ)fractio ..."
Abstract

Cited by 11 (6 self)
 Add to MetaCart
Abstract — We prove the following strong hardness result for learning: Given a distribution on labeled examples from the hypercube such that there exists a monomial (or conjunction) consistent with (1 − ϵ)fraction of the examples, it is NPhard to find a halfspace that is correct on ( 1 +ϵ)fraction of the examples, 2 for arbitrary constant ϵ> 0. In learning theory terms, weak agnostic learning of monomials by halfspaces is NPhard. This hardness result bridges between and subsumes two previous results which showed similar hardness results for the proper learning of monomials and halfspaces. As immediate corollaries of our result, we give the first optimal hardness results for weak agnostic learning of decision lists and majorities. Our techniques are quite different from previous hardness proofs for learning. We use an invariance principle and sparse approximation of halfspaces from recent work on fooling halfspaces to give a new natural list decoding of a halfspace in the context of dictatorship tests/label cover reductions. In addition, unlike previous invariance principle based proofs which are only known to give Unique Games hardness, we give a reduction from a smooth version of Label Cover that is known to be NPhard.
On agnostic boosting and parity learning
 Proceedings of the Symposium on Theory of Computing
, 2008
"... The motivating problem is agnostically learning parity functions, i.e., parity with arbitrary or adversarial noise. Specifically, given random labeled examples from an arbitrary distribution, we would like to produce an hypothesis whose accuracy nearly matches the accuracy of the best parity functio ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
The motivating problem is agnostically learning parity functions, i.e., parity with arbitrary or adversarial noise. Specifically, given random labeled examples from an arbitrary distribution, we would like to produce an hypothesis whose accuracy nearly matches the accuracy of the best parity function. Our algorithm runs in time 2 O(n / log n) , which matches the best known for the easier cases of learning parities with random classification noise (Blum et al, 2003) and for agnostically learning parities over the uniform distribution on inputs (Feldman et al, 2006). Our approach is worth noting. We give an agnostic boosting theorem that is capable of nearly achieving optimal accuracy, improving upon earlier studies (starting with Ben David et al, 2001). This is combined with an algorithm that harnesses an unexpected (very weak) agnostic ability of the (random noise) parity learning algorithm of Blum et al (2000). Our agnostic boosting framework is completely general and may be applied to other agnostic learning problems. Hence, it also sheds light on the actual difficulty of agnostic learning by showing that full agnostic boosting is indeed possible, despite previous lower bounds. 1