A Very Fast Learning Method for Neural Networks Based On Sensitivity

A Very Fast Learning Method for Neural Networks Based On Sensitivity (2006)

Cached

Download Links

[www.jmlr.org]

Other Repositories/Bibliography

DBLP

by Enrique Castillo , Bertha Guijarro-Berdinas , Oscar Fontenla-Romero , Amparo Alonso-Betanzos

Venue:	JOURNAL OF MACHINE LEARNING RESEARCH
Citations:	6 - 2 self

BibTeX

@MISC{Castillo06avery,
    author = {Enrique Castillo and Bertha Guijarro-Berdinas and Oscar Fontenla-Romero and Amparo Alonso-Betanzos},
    title = { A Very Fast Learning Method for Neural Networks Based On Sensitivity},
    year = {2006}
}

Bookmark

OpenURL

Abstract

This paper introduces a learning method for two-layer feedforward neural networks based on sensitivity analysis, which uses a linear training algorithm for each of the two layers. First, random values are assigned to the outputs of the first layer; later, these initial values are updated based on sensitivity formulas, which use the weights in each of the layers; the process is repeated until convergence. Since these

Citations

763	Numerical Methods for Unconstrained Optimization and Nonlinear Equations - Dennis - 1983
583	Learning representations by back-propagating errors - Rumelhart, Hinton, et al. - 1986
485	Deterministic nonperiodic flow - Lorenz - 1963
474	An algorithm for least-squares estimation of nonlinear parameters - MARQUARDT - 1963
283	Increased rates of convergence through learning rate adaptation - Jacobs
239	A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks - Moller - 1993
206	Nonparametric Statistical Methods - Hollander, DA - 1973
193	Learning from Data: Concepts, Theory, and Methods - Cherkassky, Mulier - 1998
192	Training feed-forward networks with the Marquardt algorithm - Hagan, Menhaj
149	Function minimization by conjugate gradients - Fletcher, Reeves - 1964
138	A Method for the Solution of Certain Non-linear Problems in Least Squares - Levenberg - 1944
99	Multiple Comparisons: Theory and Methods - Hsu - 1996
98	Convergence of the Back-Propagation Method - Vogl, Mangis, et al. - 1988
93	Efficient BackProp - LeCun, Bottou, et al. - 1998
76	Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights - Nguyen, Widrow - 1990
66	Restart procedures for the conjugate gradient method - Powell - 1977
31	Sensitivity analysis in discrete Bayesian networks - Castillo, Gutierrez, et al. - 1997
31	Speed up learning and network optimization with extended back propagation - Sperduti, Starita - 1993
27	Optimal algorithms for adaptive networks: second order back propagation, second order direct propagation, and second order Hebbian learning - Parker - 1987
26	A Derivation of Conjugate Gradients - Beale - 1972
25	Fast curvature matrix-vector products for second-order gradient descent - Schraudolph - 2002
24	First and second order methods for learning: between steepest descent and Newton's method - Battiti - 1992
22	Computing second derivatives in feed-forward networks: A review - Buntine, Weigend - 1993
22	Second order properties of error surfaces: learning time, generalization - LeCun, Kanter, et al. - 1991
18	Zunino,: Circular backpropagation networks for classification - Ridella, Rovetta, et al. - 1997
12	Using curvature information for fast stochastic search - Orr, Leen - 1997
11	Parameter adaptation in stochastic optimization - Almeida, Langlois, et al. - 1998
11	Statistically controlled activation weight initialization (scawi - Drago, Ridella - 1992
9	A global optimum approach for one-layer neural networks - Castillo, Fontenla-Romero, et al.
9	A method for self-determination of adaptive learning rates in back propagation. Neural Networks - Weir - 1991
6	A Learning Algorithm for Multilayered Neural Networks Based on Linear Least Squares Problems - Biegler-Konig, Barmann - 1993
6	Building and Solving - Castillo, Conejo, et al. - 2001
6	Improving the learning rate of back-propagation with the gradient reuse algorithm - Hush, Salas - 1989
2	Working with differential, functional and difference equations using functional networks - Castillo, Cobo, et al. - 1999
2	A general method for local sensitivity analysis with application to regression models and other optimization problems - Castillo, Hadi, et al. - 2004
1	Functional networks. a new neural network based methodology - Castillo, Cobo, et al.
1	A perturbation approach to sensitivity analysis in nonlinear programming - Castillo, and, et al.
1	Supervised learning for feed-forward neural networks: a new minimax approach for fast convergence - CASTILLO, FONTENLA-ROMERO, et al. - 1993
1	Linear least-squares based methods for neural networks learning - Fontenla-Romero, Erdogmus, et al.
1	Acceleration of learning speed in neural networks by reducing weight oscillations - Ihm, Park - 1999
1	On the peculiar distribution of the U.S. stock indeces’ first digits - Ley - 1996
1	Characterization of optical instabilities and chaos using MLP training algorithms - Pethel, Bowden, et al. - 1993
1	FONTENLA-ROMERO AND ALONSO-BETANZOS T. Tollenaere. Supersab: Fast adaptive back propagation with good scaling properties - CASTILLO - 1990
1	An algorithm for fast convergence in training neural networks - Wilamowski, Iplikci, et al. - 2001