Abstract

For neural networks with a wide class of weight-priors, it can be shown that in the limit of an infinite number of hidden units the prior over functions tends to a Gaussian process. In this paper analytic forms are derived for the covariance function of the Gaussian processes corresponding to networks with sigmoidal and Gaussian hidden units. This allows predictions to be made efficiently using networks with an infinite number of hidden units, and shows that, somewhat paradoxically, it may be easier to compute with infinite networks than finite ones.

Citations

989	Spline models for observational data - Wahba - 1990
829	Statistics for Spatial Data - Cressie - 1993
570	Networks for approximation and learning - Poggio, Girosi - 1990
346	A practical Bayesian framework for back-propagation networks - MacKay - 1992
145	Mining Geostatistics - Journel, Huijbregts - 1978
45	Prediction and Regulation: by linear least-square methods. 2nd edition - Whittle - 1984
38	Some new results on neural network approximation - Hornik - 1993
2	Bayesian Learning for Neural Netowrks - Neal - 1996