DMCA
A Theory of Networks for Approximation and Learning (1989)
| Venue: | Laboratory, Massachusetts Institute of Technology |
| Citations: | 233 - 23 self |
BibTeX
@ARTICLE{Poggio89atheory,
author = {Tomaso Poggio and Federico Girosi},
title = {A Theory of Networks for Approximation and Learning},
journal = {Laboratory, Massachusetts Institute of Technology},
year = {1989},
volume = {1140}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Learning an input-output mapping from a set of examples, of the type that many neural networks have been constructed to perform, can be regarded as synthesizing an approximation of a multi-dimensional function, that is solving the problem of hypersurface reconstruction. From this point of view, this form of learning is closely related to classical approximation techniques, such as generalized splines and regularization theory. This paper considers the problems of an exact representation and, in more detail, of the approximation of linear and nonlinear mappings in terms of simpler functions of fewer variables. Kolmogorov's theorem concerning the representation of functions of several variables in terms of functions of one variable turns out to be almost irrelevant in the context of networks for learning. Wedevelop a theoretical framework for approximation based on regularization techniques that leads to a class of three-layer networks that we call Generalized Radial Basis Functions (GRBF), since they are mathematically related to the well-known Radial Basis Functions, mainly used for strict interpolation tasks. GRBF networks are not only equivalent to generalized splines, but are also closely related to pattern recognition methods suchasParzen windows and potential functions and to several neural network algorithms, suchas Kanerva's associative memory,backpropagation and Kohonen's topology preserving map. They also haveaninteresting interpretation in terms of prototypes that are synthesized and optimally combined during the learning stage. The paper introduces several extensions and applications of the technique and discusses intriguing analogies with neurobiological data.
Keyphrases
associative memory recognition method suchasparzen window neurobiological data input-output mapping regularization technique three-layer network many neural network potential function exact representation classical approximation technique several extension several variable grbf network simpler function generalized spline multi-dimensional function hypersurface reconstruction learning stage regularization theory well-known radial basis function theoretical framework several neural network algorithm suchas kanerva nonlinear mapping radial basis function strict interpolation task
