Results 1 
2 of
2
A Complete Characterization of Statistical Query Learning with Applications to Evolvability
, 2009
"... Statistical query (SQ) learning model of Kearns is a natural restriction of the PAC learning model in which a learning algorithm is allowed to obtain estimates of statistical properties of the examples but cannot see the examples themselves [18]. We describe a new and simple characterization of the ..."
Abstract

Cited by 27 (14 self)
 Add to MetaCart
(Show Context)
Statistical query (SQ) learning model of Kearns is a natural restriction of the PAC learning model in which a learning algorithm is allowed to obtain estimates of statistical properties of the examples but cannot see the examples themselves [18]. We describe a new and simple characterization of the query complexity of learning in the SQ learning model. Unlike the previously known bounds on SQ learning [7, 9, 33, 3, 28] our characterization preserves the accuracy and the efficiency of learning. The preservation of accuracy implies that that our characterization gives the first characterization of SQ learning in the agnostic learning framework of Haussler and Kearns, Schapire and Sellie [15, 20]. The preservation of efficiency allows us to derive a new technique for the design of evolutionary algorithms in Valiant’s model of evolvability [32]. We use this technique to demonstrate the existence of a large class of monotone evolutionary learning algorithms based on square loss fitness estimation. These results differ significantly from the few known evolutionary algorithms and give evidence that evolvability in Valiant’s model is a more versatile phenomenon than there had been previous reason to suspect. 1
On the complexity of random satisfiability problems with planted solutions
 CoRR
"... For a planted satisfiability problem on n variables with k variables per constraint, the planted assignment becomes the unique solution after O(n log n) random clauses. However, the bestknown algorithms need at least nr/2 to efficiently identify any assignment weakly correlated with the planted one ..."
Abstract

Cited by 13 (5 self)
 Add to MetaCart
For a planted satisfiability problem on n variables with k variables per constraint, the planted assignment becomes the unique solution after O(n log n) random clauses. However, the bestknown algorithms need at least nr/2 to efficiently identify any assignment weakly correlated with the planted one for clause distributions that are (r − 1)wise independent (r can be as high as k). Our main result is an unconditional lower bound, tight up to logarithmic factors, of Ω̃(nr/2) clauses for statistical algorithms, a broad class of algorithms introduced in [50, 34]. We complement this with a nearly matching upper bound using a statistical algorithm. As known approaches for problems over distributions in general, and planted satisfiability problems in particular, all have statistical analogues (spectral, MCMC, gradientbased, convex optimization etc.), this provides a rigorous explanation of the large gap between the identifiability and algorithmic identifiability thresholds for random satisfiability problems with planted solutions. Our results imply that a strong form of Feige’s refutation hypothesis for averagecase SAT instances [31] holds for statistical algorithms. We also consider the closely related problem of finding the planted assignment of a random planted kCSP which is the basis of Goldreich’s proposed oneway function [41]. Our bounds extend to this problem and give concrete evidence for the security of the oneway function and the associated pseudorandom generator when used with a sufficiently hard predicate.