Abstract

We present an algorithm for improving the accuracy of algorithms for learning binary concepts. The improvement is achieved by combining a large number of hypotheses, each of which is generated by training the given learning algorithm on a different set of examples. Our algorithm is based on ideas presented by Schapire in his paper "The strength of weak learnability", and represents an improvement over his results. The analysis of our algorithm provides general upper bounds on the resources required for learning in Valiant's polynomial PAC learning framework, which are the best general upper bounds known today. We show that the number of hypotheses that are combined by our algorithm is the smallest number possible. Other outcomes of our analysis are results regarding the representational power of threshold circuits, the relation between learnability and compression, and a method for parallelizing PAC learning algorithms. We provide extensions of our algorithms to cases in which the conc...

Citations

1714	Schapire R: A decision-theoretic generalization of online learning and an application to boosting - Freund - 1997
565	Learnability and the Vapnik Chervonenkis dimension - Blumer, Ehrenfeucht, et al. - 1989
554	The strenght of weak learnability - Schapire - 1990
408	Concrete Mathematics: A Foundation for Computer Science, 2nd edition - Graham, Knuth, et al. - 1994
279	Cryptographic limitations on learning boolean formulae and finite automata - Kearns, Valiant - 1994
248	Efficient noise-tolerant learning from statistical queries - Kearns - 1998
182	Efficient distributionfree learning of probabilistic concepts - Kearns, Schapire - 1994
116	Machine Learning: A Theoretical Approach - Natarajan - 1991
91	Improving performance in neural networks using a boosting algorithm - Drucker, Schapire, et al. - 1993
89	Equivalence of models for polynomial learnability - HAUSSLER, KEARNS, et al. - 1991
80	Majority gates vs. general weighted threshold gates - Goldmann, H˚astad, et al. - 1992
55	Sample compression, learnability, and the Vapnik Chervonenkis dimension - Floyd, Warmuth - 1995
50	Predicting f0,1g functions on randomly drawn points. Information and Computation 108(2): 212{261 - Haussler, Littlestone, et al. - 1994
25	Learning boolean formulae or finite automata is as hard as factoring - Kearns, Valiant - 1988
2	private correspondence - Drucker - 1992
2	Relating data compression and learnability. This early and hard-to-locate work is referenced and partly re-written in FW93 - Littlestone, Warmuth - 1986
2	private correspondence - Schapire - 1992
2	private correspondence - Shamir - 1992