A Fast Stochastic Error-Descent Algorithm for Supervised Learning and Optimization

A Fast Stochastic Error-Descent Algorithm for Supervised Learning and Optimization (1993)

Cached

Download Links

[ftp.cis.ohio-state.edu]

Other Repositories/Bibliography

DBLP

by Gert Cauwenberghs

Venue:	In
Citations:	24 - 4 self

BibTeX

@INPROCEEDINGS{Cauwenberghs93afast,
    author = {Gert Cauwenberghs},
    title = {A Fast Stochastic Error-Descent Algorithm for Supervised Learning and Optimization},
    booktitle = {In},
    year = {1993},
    pages = {244--251},
    publisher = {Morgan Kaufmann}
}

Bookmark

OpenURL

Abstract

A parallel stochastic algorithm is investigated for error-descent learning and optimization in deterministic networks of arbitrary topology. No explicit information about internal network structure is needed. The method is based on the model-free distributed learning mechanism of Dembo and Kailath. A modified parameter update rule is proposed by which each individual parameter vector perturbation contributes a decrease in error. A substantially faster learning speed is hence allowed. Furthermore, the modified algorithm supports learning time-varying features in dynamical networks. We analyze the convergence and scaling properties of the algorithm, and present simulation results for dynamic trajectory learning in recurrent networks. 1 Background and Motivation We address general optimization tasks that require finding a set of constant parameter values p i that minimize a given error functional E(p). For supervised learning, the error functional consists of some quantitativ...

Citations

2304	Learning Internal Representations by Error Propagation - Rumelhart, Hinton, et al. - 1986
368	A Learning Algorithm for Continually Running Fully Recurrent Neural Networks’, Neural computation - Williams, Zipser - 1989
150	Learning state space trajectories in recurrent neural networks - Pearlmutter - 1989
50	30 years of adaptive neural networks - Widrow, Lehr - 1992
48	Weight perturbation: An optimal architecture and learning technique for analog vlsi feedforward and recurrent multi-layer networks - Jabri, Flower - 1992
31	A fixed size storage O(n ) time complexity learning algorithm for fully recurrent continually running networks - Schmidhuber - 1992
18	Summed Weight Neuron Perturbation: An O(N) Improvement over Weight Perturbation - Flower, Jabri - 1993
17	AParallel Gradient Descent Method for Learning in Analog VLSI Neural Networks - Alspector, Meir, et al. - 1993
16	A VLSI-efficient technique for generating multiple uncorrelated noise sources and its application to stochastic neural networks - Alspector, Gannett, et al. - 1991
12	Learning a trajectory using adjoint functions and teacher forcing. Neural Networks - Toomarian, Barhen - 1992
11	Model-free distributed learning - Dembo, Kailath - 1990
7	Analog VLSI implementation of gradient descent - Kirk, Kerns, et al. - 1993
2	Using Noise Injection and Correlation in Analog Hardware to Estimate Gradients," submitted - Anderson, Kerns - 1992
1	An Adaptive System for the Control of Aircraft and Spacecraft - Whitaker - 1959
1	Learning in Dynamical Systems: Gradient Descent, Random Descent and Modular Approaches - Baldi - 1992