The exact form of a gradient-following learning algorithm for completely recurrent networks running in continually sampled time is derived and used as the basis for practical algorithms for temporal supervised learning tasks. These algorithms have: (1) the advantage that they do not require a precisely defined training interval, operating while the network runs; and (2) the disadvantage that they require nonlocal communication in the network being trained and are computationally expensive. These algorithms are shown to allow networks having recurrent connections to learn complex tasks requiring the retention of information over time periods having either fixed or indefinite length. 1 Introduction A major problem in connectionist theory is to develop learning algorithms that can tap the full computational power of neural networks. Much progress has been made with feedforward networks, and attention has recently turned to developing algorithms for networks with recurrent connections, wh...

Introduction 1.1 Learning in Recurrent Networks Connectionist networks having feedback connections are interesting for a number of reasons. Biological neural networks are highly recurrently connected, and many authors have studied recurrent network models of various types of perceptual and memory processes. The general property making such networks interesting and potentially useful is that they manifest highly nonlinear dynamical behavior. One such type of dynamical behavior that has received much attention is that of settling to a fixed stable state, but probably of greater importance both biologically and from an engineering viewpoint are time-varying behaviors. Here we consider algorithms for training recurrent networks to perform temporal supervised learning tasks, in which the specification of desired behavior is in the form of specific examples of input and desired output trajectories. One example of such a task is sequence classification, where

This paper starts by placing neural net techniques in a general nonlinear control framework. After that, several basic theoretical results on networks are surveyed. 1

The extended Kalman filter (EKF) can be used as an on-line algorithm to determine the weights in a recurrent network given target outputs as it runs. This paper notes some relationships between the EKF as applied to recurrent net learning and some simpler techniques that are more widely used. In particular, making certain simplifications to the EKF gives rise to an algorithm essentially identical to the real-time recurrent learning (RTRL) algorithm. Since the EKF involves adjusting unit activity in the network, it also provides a principled generalization of the teacher forcing technique. Prelinary simulation experiments on simple finite-state Boolean tasks indicate that the EKF can provide substantial speed-up in number of time steps required for training on such problems when compared with simpler on-line gradient algorithms. The computational requirements of the EKF are steep, but turn out to scale with network size at the same rate as RTRL. These observations are intended to provid...

Introduction The purpose of this chapter is to provide an introductory overview of some of the current research efforts directed toward adapting the weights in connectionist networks having feedback connections. While much of the recent emphasis in the field has been on multilayer networks having no such feedback connections, it is likely that the use of recurrently connected networks will be of particular importance for applications to the control of dynamical systems. Following the approach taken in the previous chapter by Andy Barto, this chapter will emphasize the relationship of connectionist research in this area to strategies used in more conventional engineering circles for modelling and controlling dynamical systems, while at the same time noting what there is in the connectionist approach that is novel. In particular, I will argue that while much of the connectionist approach to adapting the weights in recurrent networks having interesting dynamics rests on the same

This report constitutes an expanded version of a presentation given by the author at the 1993 European Control Conference (short course on "Neural Nets for Control"). The first part places neurocontrol techniques in a general learning control framework. The second part of the report, which is essentially independent of the first, briefly surveys several basic theoretical results regarding neural networks.

The extended Kalman filter (EKF) can be used as an on-line algorithm to determine the weights in a recurrent network given target outputs as it runs. This involves forming an augmented network state vector consisting of all unit activities and weights. This report notes some relationships between the EKF as applied to recurrent net learning and some simpler techniques that are more widely used. In particular, it is shown that making certain simplifications to the EKF gives rise to an algorithm essentially identical to the real-time recurrent learning (RTRL) algorithm. That is, the resulting algorithm both maintains the RTRL data structure and prescribes identical weight changes. In addition, because the EKF also involves adjusting unit activity in the network, it provides a principled generalization of the useful "teacher forcing" technique. Very preliminary experiments on simple finitestate Boolean tasks indicate that the EKF works well for these, generally giving substantial speed-up...