Abstract

. We consider learning on multi-layer neural nets with piecewise polynomial activation functions and a fixed number k of numerical inputs. We exhibit arbitrarily large network architectures for which efficient and provably successful learning algorithms exist in the rather realistic refinement of Valiant's model for probably approximately correct learning ("PAC-learning") where no a-priori assumptions are required about the "target function" (agnostic learning), arbitrary noise is permitted in the training sample, and the target outputs as well as the network outputs may be arbitrary reals. The number of computation steps of the learning algorithm LEARN that we construct is bounded by a polynomial in the bit-length n of the fixed number of input variables, in the bound s for the allowed bit-length of weights, in 1 " , where " is some arbitrary given bound for the true error of the neural net after training, and in 1 ffi where ffi is some arbitrary given bound for the probability t...

Citations

1515	A theory of the learnable - Valiant - 1984
1055	Combinatorial Optimization: Algorithms and Complexity - Papadimitriou, Steiglitz - 1982
565	Learnability and the Vapnik Chervonenkis dimension - Blumer, Ehrenfeucht, et al. - 1989
399	An introduction to computing with neural nets - Lippmann - 1987
377	Parallel distributed processing: Explorations in the microstructure of cognition - Rumelhart, McClelland - 1986
327	Decision theoretic generalizations of the PAC model for neural net and other learning applications - Haussler - 1992
296	What size net gives valid generalization - Baum, Haussler - 1989
279	Cryptographic limitations on learning boolean formulae and finite automata - Kearns, Valiant - 1994
186	Training a 3-node neural networks is np-complete - Blum, Rivest - 1992
169	Toward efficient agnostic learning - KEARNS, SCHAPIRE, et al. - 1994
113	On the Betti Numbers of Real Varieties - Milnor - 1964
89	Bounding the Vapnik-Chervonenkis dimension of concept classes parameterized by real numbers - Goldberg, Jerrum - 1995
87	Neural Network design and the complexity of learning - Judd - 1990
83	Empirical processes: theory and applications - Pollard - 1990
73	The Complexity of Computing - Savage - 1976
59	Bounds for the computational power and learning complexity of analog neural nets - Maass - 1997
56	On the computational complexity and geometry of the first order theory of the reals - Renegar - 1992
29	Vapnik-Chervonenkis dimension of neural nets - Maass - 1995
26	Capacity problems for linear machines - Cover - 1968
18	Perspectives of current research about the complexity of learning in neural nets - Maass - 1994
5	Efficient distribution free learning of probabilistic concepts - Kearns, Schapire - 1990
2	Computing on Analog Neural Nets with Arbitrary Real Weights - Maass - 1994
1	Schapire, "Equivalence of models for polynomial learnability - Haussler, Kearns, et al. - 1991