Results 1  10
of
15
Learning longterm dependencies in NARX recurrent neural networks
, 1996
"... It has recently been shown that gradientdescent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. We show tht the longterm de ..."
Abstract

Cited by 47 (5 self)
 Add to MetaCart
It has recently been shown that gradientdescent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. We show tht the longterm dependencies problem is lessened for a class of architectures called NARX recurrent neural networks, which have powerful representational capabilities. We have previously reported that gradient descent learning can be more effective in NARX networks than in recurrent neural network architectures that have "hidden states" on problems including grammatical inference and nonlinear system identification. Typically, the network converges much faster and generalizes better than other networks. The results in this paper are consistent with this phenomenon. We present some experimental results which show that NARX networks can often retain information for two to three times as long as conventi...
Computational Capabilities of Recurrent NARX Neural Networks
 IEEE Trans. on Systems, Man and Cybernetics
, 1997
"... Abstract—Recently, fully connected recurrent neural networks have been proven to be computationally rich—at least as powerful as Turing machines. This work focuses on another network which is popular in control applications and has been found to be very effective at learning a variety of problems. T ..."
Abstract

Cited by 31 (8 self)
 Add to MetaCart
Abstract—Recently, fully connected recurrent neural networks have been proven to be computationally rich—at least as powerful as Turing machines. This work focuses on another network which is popular in control applications and has been found to be very effective at learning a variety of problems. These networks are based upon Nonlinear AutoRegressive models with eXogenous Inputs (NARX models), and are therefore called NARX networks. As opposed to other recurrent networks, NARX networks have a limited feedback which comes only from the output neuron rather than from hidden states. They are formalized by y(t) =9(u(t0nu);111;u(t01); u(t);y(t0ny);111;y(t01)) where u(t) and y(t) represent input and output of the network at time t, nu and ny are the input and output order, and the function 9 is the mapping performed by a Multilayer Perceptron. We constructively prove that the NARX networks with a finite number of parameters are computationally as strong as fully connected recurrent networks and thus Turing machines. We conclude that in theory one can use the NARX models, rather than conventional recurrent networks without any computational loss even though their feedback is limited. Furthermore, these results raise the issue of what amount of feedback or recurrence is necessary for any network to be Turing equivalent and what restrictions on feedback limit computational power. I.
Learning longterm dependencies is not as difficult with NARX recurrent neural networks
, 1996
"... It has recently been shown that gradient descent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. In this paper we explore the lon ..."
Abstract

Cited by 18 (3 self)
 Add to MetaCart
It has recently been shown that gradient descent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. In this paper we explore the longterm dependencies problem for a class of architectures called NARX recurrent neural networks, which have powerful representational capabilities. We have previously reported that gradient descent learning is more effective in NARX networks than in recurrent neural network architectures that have "hidden states" on problems including grammatical inference and nonlinear system identification. Typically, the network converges much faster and generalizes better than other networks. The results in this paper are an attempt to explain this phenomenon. We present some experimental results which show that NARX networks can often retain information for two to three times as long as conventional rec...
A delay damage model selection algorithm for NARX neural networks
 IEEE TRANSACTIONS ON SIGNAL PROCESSING
, 1997
"... Recurrent neural networks have become popular models for system identification and time series prediction. Nonlinear autoregressive models with exogenous inputs (NARX) neural network models are a popular subclass of recurrent networks and have been used in many applications. Although embedded memory ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
Recurrent neural networks have become popular models for system identification and time series prediction. Nonlinear autoregressive models with exogenous inputs (NARX) neural network models are a popular subclass of recurrent networks and have been used in many applications. Although embedded memory can be found in all recurrent network models, it is particularly prominent in NARX models. We show that using intelligent memory order selection through pruning and good initial heuristics significantly improves the generalization and predictive performance of these nonlinear systems on problems as diverse as grammatical inference and time series prediction.
Optimization and Simulation of Quality Properties in Paper Machine with Neural Networks
 Proc. of IEEE International Conference on Neural Networks
, 1994
"... The final quality of paper depends on many quality and process variables. It is very difficult to find theoretical rules of the behavior of paper properties when variables depend from each other and when the interdependencies are not linear. In this paper we present a neural network based system for ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
The final quality of paper depends on many quality and process variables. It is very difficult to find theoretical rules of the behavior of paper properties when variables depend from each other and when the interdependencies are not linear. In this paper we present a neural network based system for estimating the final quality of paper from process measurements. Inverse computation of the network model is used to find a control action that will produce the desired quality. A separate selforganizing map is used to monitor the movement of the operating point of the process and to give a hint of the estimation error of the network. I. Introduction The final quality of paper depends on many quality and process variables in a complicated and nonlinear way. It is very difficult to find a physical model that would describe the process so accurately, that the model could be used to instruct the machine operators in everyday control actions (i.e., how to change the adjustable process parame...
Online Learning in Adaptive Neurocontrol Schemes with a Sliding Mode Algorithm
"... Abstract—The novel features of an adaptive PIDlike neurocontrol scheme for nonlinear plants are presented. The controller tuning is based on an estimate of the commanderror on its output by using a neural predictive model. A robust online learning algorithm, based on the direct use of sliding mode ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
Abstract—The novel features of an adaptive PIDlike neurocontrol scheme for nonlinear plants are presented. The controller tuning is based on an estimate of the commanderror on its output by using a neural predictive model. A robust online learning algorithm, based on the direct use of sliding mode control (SMC) theory is applied. The proposed approach allows handling of the plant–model mismatches, uncertainties and parameters changes. The results show that both the plant model and the controller inherit some of the advantages of SMC, such as high speed of learning and robustness. Index Terms—Adaptive control, intelligent control, learning control systems, neurocontrollers, variable structure systems. I.
Nonlinear predictive control based on neural multimodels
 International Journal of Applied Mathematics and Computer Science 20(1): 7–21, DOI
, 2010
"... This paper discusses neural multimodels based on Multi Layer Perceptron (MLP) networks and a computationally efficient nonlinear Model Predictive Control (MPC) algorithm which uses such models. Thanks to the nature of the model it calculates future predictions without using previous predictions. Th ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
This paper discusses neural multimodels based on Multi Layer Perceptron (MLP) networks and a computationally efficient nonlinear Model Predictive Control (MPC) algorithm which uses such models. Thanks to the nature of the model it calculates future predictions without using previous predictions. This means that, unlike the classical Nonlinear Auto Regressive with eXternal input (NARX) model, the multimodel is not used recurrently in MPC, and the prediction error is not propagated. In order to avoid nonlinear optimisation, in the discussed suboptimal MPC algorithm the neural multimodel is linearised online and, as a result, the future control policy is found by solving of a quadratic programming problem.
Dynamic Routing of Telephone Traffic Using . . .
, 1996
"... In this thesis, the consequences of automating network management of telephone networks are examined. The role of network managers is to monitor the network for exceptional conditions and place controls into the network if necessary to deal with these network exceptions. One potential consequence ..."
Abstract
 Add to MetaCart
In this thesis, the consequences of automating network management of telephone networks are examined. The role of network managers is to monitor the network for exceptional conditions and place controls into the network if necessary to deal with these network exceptions. One potential consequence of automating network management is a network which is capable of adjusting itself quickly to changing tra c conditions, also known as a network with dynamic routing. Simulations are used to show that there are bene ts to be gained from implementing dynamic routing by automating the actions of the network managers. In this thesis, the application of learning techniques such as neural networks and linear predictors to the tasks of network tra c management is also examined. Three network management tasks considered are: (i) recognition of tra c patterns in the network (ii) learning suitable thresholds for network congestion control and (iii) time series prediction of trunk group occupancy. It is found that nonlinear learning techniques such as neural networks can give small gains over the more standard technique of linear predictors.
Autonomic Adaptation in Distributed Systems and Networks
"... There is now considerable interest in industry and academia for “autonomic” or selfadaptive networks and distributed systems. Largescale applications, such as simulation, typically run on large distributed systems in which it is impossible to guarantee at each instant the full reliability and avai ..."
Abstract
 Add to MetaCart
There is now considerable interest in industry and academia for “autonomic” or selfadaptive networks and distributed systems. Largescale applications, such as simulation, typically run on large distributed systems in which it is impossible to guarantee at each instant the full reliability and availability of all processing nodes and of all communication links. Nevertheless such systems have to accomplish their mission and provide a high level of dependability and the best possible performance to critical applications. Selfadaptation will therefore be an intrinsic feature of large distributed systems and networks in order to move tasks and files in response to changes in subsystem dependability and system load, and to provide QualityofService and dependable connections in the presence of fluctuating workloads and unknown system behavior. In this paper we review our work on the design of adaptive online task management algorithms for distributed systems, and the control of cognitive packet networks (CPN) to offer user specified QoS.
DYNAMIC SYSTEM MODELLING USING RULES3 INDUCTION ALGORITHM
"... Abstract This paper describes the use of a new induction algorithm to derive production rules for modelling dynamic systems. The algorithm, called RULES3, is an efficient tool for extracting a compact set of IFTHEN rules from a collection of examples to classify objects into known categories. The ..."
Abstract
 Add to MetaCart
Abstract This paper describes the use of a new induction algorithm to derive production rules for modelling dynamic systems. The algorithm, called RULES3, is an efficient tool for extracting a compact set of IFTHEN rules from a collection of examples to classify objects into known categories. The paper summarises the operation of RULES3 and presents the results obtained in the modelling of two linear systems and two nonlinear systems. Key words System Identifıcation, Inductive Learning, RULES3. 1.