DMCA
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks (1989)
| Citations: | 534 - 4 self |
BibTeX
@MISC{Williams89alearning,
author = {Ronald J. Williams and David Zipser},
title = {A Learning Algorithm for Continually Running Fully Recurrent Neural Networks},
year = {1989}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
The exact form of a gradient-following learning algorithm for completely recurrent networks running in continually sampled time is derived and used as the basis for practical algorithms for temporal supervised learning tasks. These algorithms have: (1) the advantage that they do not require a precisely defined training interval, operating while the network runs; and (2) the disadvantage that they require nonlocal communication in the network being trained and are computationally expensive. These algorithms are shown to allow networks having recurrent connections to learn complex tasks requiring the retention of information over time periods having either fixed or indefinite length. 1 Introduction A major problem in connectionist theory is to develop learning algorithms that can tap the full computational power of neural networks. Much progress has been made with feedforward networks, and attention has recently turned to developing algorithms for networks with recurrent connections, wh...
Keyphrases
learning algorithm recurrent connection recurrent network nonlocal communication gradient-following learning algorithm connectionist theory sampled time much progress feedforward network defined training interval practical algorithm exact form complex task time period indefinite length learning task neural network full computational power major problem
