Statistical Ideas for Selecting Network Architectures

Statistical Ideas for Selecting Network Architectures (1995)

Cached

Download Links

[www.stats.ox.ac.uk]

by B. D. Ripley

Venue:	Invited Presentation, Neural Information Processing Systems 8
Citations:	16 - 2 self

BibTeX

@INPROCEEDINGS{Ripley95statisticalideas,
    author = {B. D. Ripley},
    title = {Statistical Ideas for Selecting Network Architectures},
    booktitle = {Invited Presentation, Neural Information Processing Systems 8},
    year = {1995},
    pages = {183--190},
    publisher = {Springer}
}

Bookmark

OpenURL

Abstract

Choosing the architecture of a neural network is one of the most important problems in making neural networks practically useful, but accounts of applications usually sweep these details under the carpet. How many hidden units are needed? Should weight decay be used, and if so how much? What type of output units should be chosen? And so on. We address these issues within the framework of statistical theory for model choice, which provides a number of workable approximate answers. This paper is principally concerned with architecture selection issues for feed-forward neural networks (also known as multi-layer perceptrons). Many of the same issues arise in selecting radial basis function networks, recurrent networks and more widely. These problems occur in a much wider context within statistics, and applied statisticians have been selecting and combining models for decades. Two recent discussions are [4, 5]. References [3, 20, 21, 22] discuss neural networks from a statistical perspecti...

Citations

896	BD: Pattern Recognition and Neural Networks - Ripley - 1996
887	Bayesian Data Analysis - Gelman, Carlin, et al. - 2004
559	Cross-validatory choice and assessment of statistical predictions - STONE - 1974
320	PJ: The behavior of maximum likelihood estimates under nonstandard conditions - Huber
267	When networks disagree: ensemble methods for hybrid neural networks - Perrone, Cooper - 1993
226	Nonlinear Regression - Seber, Wild - 1989
213	Model selection and accounting for model uncertainty in graphical models using Occam’s window - Madigan, Raftery - 1994
166	Bayesian graphical models for discrete data - Madigan, York - 1995
159	The effective number of parameters: an analysis of generalization and regularization in nonlinear learning systems”, in eds - Moody - 1992
127	Neural networks: a review from a statistical perspective. With comments and a rejoinder by the authors - Cheng, Titterington - 1994
120	Network information criterion - determining the number of hidden units for an artificial neural network model - Murata, Yoshizawa, et al. - 1994
119	Neural networks and related methods for classification - Ripley - 1994
114	D: Stacked Generalization. Neural Networks - Wolpert - 1992
105	Bayesian computation and stochastic systems - Besag, Green, et al. - 1995
99	Assessment and propagation of model uncertainty (with discussion - Draper - 1995
82	Statistical Aspects of Neural Networks - Ripley - 1993
77	An asymptotic equivalence of choice of model by crossvalidation and Akaike’s criterion - Stone - 1977
75	Neural networks and the bias/variance dilemma. Neural Comput - Geman, Bienenstock, et al. - 1992
54	Synergy of clustering multiple back-propagation networks - Lincoln, Skrzypek - 1990
45	Utans J: Principled Architecture Selection for Neural Networks: Application to Corporate Bond Rating Prediction - Moody
40	Theory of Probability, Third edition - Jeffreys - 1961
39	Improving the Generalization Properties of Radial Basis Function Neural Networks - Bishop - 1991
37	Note on generalization, regularization, and architecture selection in nonlinear learning systems - Moody - 1991
32	Flexible non-linear approaches to classification,” in From Statistics to Neural Networks - Ripley - 1994
28	Architecture Selection Strategies for Neural Networks: Application to Bond Rating Prediction - Moody, Utans - 1995
19	A completely automatic French curve - Wahba, Wold - 1975
12	Hierarchical Bayesian Analysis Using Monte Carlo Integration: Computing Posterior Distributions When There are Many Possible Models - Stewart - 1987
9	Learning curves, model selection and complexity of neural networks - Murata, Yoshizawa, et al. - 1993
7	A criterion for determining the number of parameters in an artificial neural network model - Murata, Yoshizawa, et al. - 1991
6	Neural networks and flexible regression and discrimination - Ripley - 1994