Results 1  10
of
12
Learning longterm dependencies in NARX recurrent neural networks
, 1996
"... It has recently been shown that gradientdescent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. We show tht the longterm de ..."
Abstract

Cited by 46 (5 self)
 Add to MetaCart
It has recently been shown that gradientdescent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. We show tht the longterm dependencies problem is lessened for a class of architectures called NARX recurrent neural networks, which have powerful representational capabilities. We have previously reported that gradient descent learning can be more effective in NARX networks than in recurrent neural network architectures that have "hidden states" on problems including grammatical inference and nonlinear system identification. Typically, the network converges much faster and generalizes better than other networks. The results in this paper are consistent with this phenomenon. We present some experimental results which show that NARX networks can often retain information for two to three times as long as conventi...
Computational Capabilities of Recurrent NARX Neural Networks
 IEEE Trans. on Systems, Man and Cybernetics
, 1997
"... Abstract—Recently, fully connected recurrent neural networks have been proven to be computationally rich—at least as powerful as Turing machines. This work focuses on another network which is popular in control applications and has been found to be very effective at learning a variety of problems. T ..."
Abstract

Cited by 31 (8 self)
 Add to MetaCart
Abstract—Recently, fully connected recurrent neural networks have been proven to be computationally rich—at least as powerful as Turing machines. This work focuses on another network which is popular in control applications and has been found to be very effective at learning a variety of problems. These networks are based upon Nonlinear AutoRegressive models with eXogenous Inputs (NARX models), and are therefore called NARX networks. As opposed to other recurrent networks, NARX networks have a limited feedback which comes only from the output neuron rather than from hidden states. They are formalized by y(t) =9(u(t0nu);111;u(t01); u(t);y(t0ny);111;y(t01)) where u(t) and y(t) represent input and output of the network at time t, nu and ny are the input and output order, and the function 9 is the mapping performed by a Multilayer Perceptron. We constructively prove that the NARX networks with a finite number of parameters are computationally as strong as fully connected recurrent networks and thus Turing machines. We conclude that in theory one can use the NARX models, rather than conventional recurrent networks without any computational loss even though their feedback is limited. Furthermore, these results raise the issue of what amount of feedback or recurrence is necessary for any network to be Turing equivalent and what restrictions on feedback limit computational power. I.
Learning longterm dependencies is not as difficult with NARX recurrent neural networks
, 1996
"... It has recently been shown that gradient descent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. In this paper we explore the lon ..."
Abstract

Cited by 18 (3 self)
 Add to MetaCart
It has recently been shown that gradient descent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. In this paper we explore the longterm dependencies problem for a class of architectures called NARX recurrent neural networks, which have powerful representational capabilities. We have previously reported that gradient descent learning is more effective in NARX networks than in recurrent neural network architectures that have "hidden states" on problems including grammatical inference and nonlinear system identification. Typically, the network converges much faster and generalizes better than other networks. The results in this paper are an attempt to explain this phenomenon. We present some experimental results which show that NARX networks can often retain information for two to three times as long as conventional rec...
A delay damage model selection algorithm for NARX neural networks
 IEEE TRANSACTIONS ON SIGNAL PROCESSING
, 1997
"... Recurrent neural networks have become popular models for system identification and time series prediction. Nonlinear autoregressive models with exogenous inputs (NARX) neural network models are a popular subclass of recurrent networks and have been used in many applications. Although embedded memory ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
Recurrent neural networks have become popular models for system identification and time series prediction. Nonlinear autoregressive models with exogenous inputs (NARX) neural network models are a popular subclass of recurrent networks and have been used in many applications. Although embedded memory can be found in all recurrent network models, it is particularly prominent in NARX models. We show that using intelligent memory order selection through pruning and good initial heuristics significantly improves the generalization and predictive performance of these nonlinear systems on problems as diverse as grammatical inference and time series prediction.
Optimization and Simulation of Quality Properties in Paper Machine with Neural Networks
 Proc. of IEEE International Conference on Neural Networks
, 1994
"... The final quality of paper depends on many quality and process variables. It is very difficult to find theoretical rules of the behavior of paper properties when variables depend from each other and when the interdependencies are not linear. In this paper we present a neural network based system for ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
The final quality of paper depends on many quality and process variables. It is very difficult to find theoretical rules of the behavior of paper properties when variables depend from each other and when the interdependencies are not linear. In this paper we present a neural network based system for estimating the final quality of paper from process measurements. Inverse computation of the network model is used to find a control action that will produce the desired quality. A separate selforganizing map is used to monitor the movement of the operating point of the process and to give a hint of the estimation error of the network. I. Introduction The final quality of paper depends on many quality and process variables in a complicated and nonlinear way. It is very difficult to find a physical model that would describe the process so accurately, that the model could be used to instruct the machine operators in everyday control actions (i.e., how to change the adjustable process parame...
Online Learning in Adaptive Neurocontrol Schemes with a Sliding Mode Algorithm
"... Abstract—The novel features of an adaptive PIDlike neurocontrol scheme for nonlinear plants are presented. The controller tuning is based on an estimate of the commanderror on its output by using a neural predictive model. A robust online learning algorithm, based on the direct use of sliding mode ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
Abstract—The novel features of an adaptive PIDlike neurocontrol scheme for nonlinear plants are presented. The controller tuning is based on an estimate of the commanderror on its output by using a neural predictive model. A robust online learning algorithm, based on the direct use of sliding mode control (SMC) theory is applied. The proposed approach allows handling of the plant–model mismatches, uncertainties and parameters changes. The results show that both the plant model and the controller inherit some of the advantages of SMC, such as high speed of learning and robustness. Index Terms—Adaptive control, intelligent control, learning control systems, neurocontrollers, variable structure systems. I.
Nonlinear predictive control based on neural multimodels
 International Journal of Applied Mathematics and Computer Science 20(1): 7–21, DOI
, 2010
"... This paper discusses neural multimodels based on Multi Layer Perceptron (MLP) networks and a computationally efficient nonlinear Model Predictive Control (MPC) algorithm which uses such models. Thanks to the nature of the model it calculates future predictions without using previous predictions. Th ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
This paper discusses neural multimodels based on Multi Layer Perceptron (MLP) networks and a computationally efficient nonlinear Model Predictive Control (MPC) algorithm which uses such models. Thanks to the nature of the model it calculates future predictions without using previous predictions. This means that, unlike the classical Nonlinear Auto Regressive with eXternal input (NARX) model, the multimodel is not used recurrently in MPC, and the prediction error is not propagated. In order to avoid nonlinear optimisation, in the discussed suboptimal MPC algorithm the neural multimodel is linearised online and, as a result, the future control policy is found by solving of a quadratic programming problem.
Genetic Algorithm Based Recurrent Fuzzy Neural Network Modeling of Chemical Processes
"... Abstract: A genetic algorithm (GA) based recurrent fuzzy neural network modeling method for dynamic nonlinear chemical process is presented. The dynamic recurrent fuzzy neural network (RFNN) is constructed in terms of TakagiSugeno fuzzy model. The consequent part is comprised of the dynamic neurons ..."
Abstract
 Add to MetaCart
Abstract: A genetic algorithm (GA) based recurrent fuzzy neural network modeling method for dynamic nonlinear chemical process is presented. The dynamic recurrent fuzzy neural network (RFNN) is constructed in terms of TakagiSugeno fuzzy model. The consequent part is comprised of the dynamic neurons with output feedback. The number and the parameters of membership functions in the premise part are optimized by the GA considering both the approximation capability and structure complexity of RFNN. The proposed dynamic model is applied to a PH neutralization process and the advantages of the resulting model are demonstrated.
Intelligent Substation Project Team
, 2004
"... Information about this project For information about this project contact: ..."
Abstract
 Add to MetaCart
Information about this project For information about this project contact:
Nonlinear Adaptive Prediction of Speech with a Pipelined Recurrent Neural Network and Advanced Learning Algorithms
"... New learning algorithms for an adaptive nonlinear forward predictor which is based on a Pipelined Recurrent Neural Network (PRNN) are presented. A computationally efficient Gradient Descent (GD) algorithm, as well as a novel Extended Recursive Least Squares (ERLS) algorithm are tested on the pre ..."
Abstract
 Add to MetaCart
New learning algorithms for an adaptive nonlinear forward predictor which is based on a Pipelined Recurrent Neural Network (PRNN) are presented. A computationally efficient Gradient Descent (GD) algorithm, as well as a novel Extended Recursive Least Squares (ERLS) algorithm are tested on the predictor. Simulation studies, based on three speech signals, which have been made public and are available on the World Wide Web (WWW), show that the nonlinear predictor does not perform satisfactorily when the previously proposed gradient descent algorithm was used. The steepest descent algorithm is shown to yield a poor performance in terms of the prediction error gain, whereas consistently improved results are obtained using the ERLS algorithm. The merit of the nonlinear predictor structure is confirmed by yielding approximately 2 dB higher prediction gain than only a linear structure predictor, which uses the conventional Recursive Least Squares (RLS) algorithm. 1.1 Introduction...