Abstract

In this paper we study learning in the PAC model of Valiant [18] in which the example oracle used for learning may be faulty in one of two ways: either by misclassifying the example or by distorting the distribution of examples. We first consider models in which examples are misclassified. Kearns [12] recently showed that efficient learning in a new model using statistical queries is a sufficient condition for PAC learning with classification noise. We show that efficient learning with statistical queries is sufficient for learning in the PAC model with malicious error rate proportional to the required statistical query accuracy. One application of this result is a new lower bound for tolerable malicious error in learning monomials of k literals. This is the first such bound which is independent of the number of irrelevant attributes n. We also use the statistical query model to give sufficient conditions for using distribution specific algorithms on distributions outside their prescr...

Citations

1515	A Theory of the Learnable - Valiant - 1984
554	The strenght of weak learnability - Schapire - 1990
253	Constant Depth Circuits, Fourier Transform, and Learnability - Linial, Mansour, et al. - 1993
248	Efficient noise-tolerant learning from statistical queries - Kearns - 1998
207	Quantifying inductive bias: AI learning algorithms and Valiant’s learning framework - Haussler - 1988
197	Learning from noisy examples - Angluin, Laird - 1988
158	Learning in the presence of malicious errors - Kearns, Li - 1993
155	On the Learnability of Boolean Formula - Kearns, Li, et al. - 1987
114	Learning disjunctions of conjunctions - Valiant - 1985
56	The Computational Complexity of Machine Learning - Kearns - 1990
47	An Improved Boosting Algorithm and its Implications on Learning Complexity - Freund - 1992
41	On the necessity of Occam algorithms - Board, Pitt - 1992
39	Learnability by fixed distributions - Benedek, Itai - 1988
24	Learning 2�� DNF formulas and k�� decision trees - Hancock - 1991
17	Improved learning of AC ° functions - Furst, Jackson, et al. - 1991
12	Investigating the distributional assumptions of the pac learning model - Bartlett, Williamson - 1991
12	Learning by distances - Ben-David, Itai, et al. - 1995
11	Learning with a slowly changing distribution - Bartlett - 1992
6	Dominating distributions and learnability - Benedek, Itai - 1992