Results 1  10
of
17
Learning longterm dependencies in NARX recurrent neural networks
, 1996
"... It has recently been shown that gradientdescent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. We show tht the longterm de ..."
Abstract

Cited by 65 (5 self)
 Add to MetaCart
It has recently been shown that gradientdescent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. We show tht the longterm dependencies problem is lessened for a class of architectures called NARX recurrent neural networks, which have powerful representational capabilities. We have previously reported that gradient descent learning can be more effective in NARX networks than in recurrent neural network architectures that have "hidden states" on problems including grammatical inference and nonlinear system identification. Typically, the network converges much faster and generalizes better than other networks. The results in this paper are consistent with this phenomenon. We present some experimental results which show that NARX networks can often retain information for two to three times as long as conventi...
Computational Capabilities of Recurrent NARX Neural Networks
 IEEE Trans. on Systems, Man and Cybernetics
, 1997
"... Abstract—Recently, fully connected recurrent neural networks have been proven to be computationally rich—at least as powerful as Turing machines. This work focuses on another network which is popular in control applications and has been found to be very effective at learning a variety of problems. T ..."
Abstract

Cited by 42 (9 self)
 Add to MetaCart
(Show Context)
Abstract—Recently, fully connected recurrent neural networks have been proven to be computationally rich—at least as powerful as Turing machines. This work focuses on another network which is popular in control applications and has been found to be very effective at learning a variety of problems. These networks are based upon Nonlinear AutoRegressive models with eXogenous Inputs (NARX models), and are therefore called NARX networks. As opposed to other recurrent networks, NARX networks have a limited feedback which comes only from the output neuron rather than from hidden states. They are formalized by y(t) =9(u(t0nu);111;u(t01); u(t);y(t0ny);111;y(t01)) where u(t) and y(t) represent input and output of the network at time t, nu and ny are the input and output order, and the function 9 is the mapping performed by a Multilayer Perceptron. We constructively prove that the NARX networks with a finite number of parameters are computationally as strong as fully connected recurrent networks and thus Turing machines. We conclude that in theory one can use the NARX models, rather than conventional recurrent networks without any computational loss even though their feedback is limited. Furthermore, these results raise the issue of what amount of feedback or recurrence is necessary for any network to be Turing equivalent and what restrictions on feedback limit computational power. I.
Learning longterm dependencies is not as difficult with NARX recurrent neural networks
, 1996
"... It has recently been shown that gradient descent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. In this paper we explore the lon ..."
Abstract

Cited by 18 (3 self)
 Add to MetaCart
It has recently been shown that gradient descent learning algorithms for recurrent neural networks can perform poorly on tasks that involve longterm dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. In this paper we explore the longterm dependencies problem for a class of architectures called NARX recurrent neural networks, which have powerful representational capabilities. We have previously reported that gradient descent learning is more effective in NARX networks than in recurrent neural network architectures that have "hidden states" on problems including grammatical inference and nonlinear system identification. Typically, the network converges much faster and generalizes better than other networks. The results in this paper are an attempt to explain this phenomenon. We present some experimental results which show that NARX networks can often retain information for two to three times as long as conventional rec...
A delay damage model selection algorithm for NARX neural networks
 IEEE TRANSACTIONS ON SIGNAL PROCESSING
, 1997
"... Recurrent neural networks have become popular models for system identification and time series prediction. Nonlinear autoregressive models with exogenous inputs (NARX) neural network models are a popular subclass of recurrent networks and have been used in many applications. Although embedded memory ..."
Abstract

Cited by 13 (1 self)
 Add to MetaCart
(Show Context)
Recurrent neural networks have become popular models for system identification and time series prediction. Nonlinear autoregressive models with exogenous inputs (NARX) neural network models are a popular subclass of recurrent networks and have been used in many applications. Although embedded memory can be found in all recurrent network models, it is particularly prominent in NARX models. We show that using intelligent memory order selection through pruning and good initial heuristics significantly improves the generalization and predictive performance of these nonlinear systems on problems as diverse as grammatical inference and time series prediction.
Optimization and Simulation of Quality Properties in Paper Machine with Neural Networks
 Proc. of IEEE International Conference on Neural Networks
, 1994
"... The final quality of paper depends on many quality and process variables. It is very difficult to find theoretical rules of the behavior of paper properties when variables depend from each other and when the interdependencies are not linear. In this paper we present a neural network based system for ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
The final quality of paper depends on many quality and process variables. It is very difficult to find theoretical rules of the behavior of paper properties when variables depend from each other and when the interdependencies are not linear. In this paper we present a neural network based system for estimating the final quality of paper from process measurements. Inverse computation of the network model is used to find a control action that will produce the desired quality. A separate selforganizing map is used to monitor the movement of the operating point of the process and to give a hint of the estimation error of the network. I. Introduction The final quality of paper depends on many quality and process variables in a complicated and nonlinear way. It is very difficult to find a physical model that would describe the process so accurately, that the model could be used to instruct the machine operators in everyday control actions (i.e., how to change the adjustable process parame...
Online Learning in Adaptive Neurocontrol Schemes with a Sliding Mode Algorithm
"... Abstract—The novel features of an adaptive PIDlike neurocontrol scheme for nonlinear plants are presented. The controller tuning is based on an estimate of the commanderror on its output by using a neural predictive model. A robust online learning algorithm, based on the direct use of sliding mode ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
(Show Context)
Abstract—The novel features of an adaptive PIDlike neurocontrol scheme for nonlinear plants are presented. The controller tuning is based on an estimate of the commanderror on its output by using a neural predictive model. A robust online learning algorithm, based on the direct use of sliding mode control (SMC) theory is applied. The proposed approach allows handling of the plant–model mismatches, uncertainties and parameters changes. The results show that both the plant model and the controller inherit some of the advantages of SMC, such as high speed of learning and robustness. Index Terms—Adaptive control, intelligent control, learning control systems, neurocontrollers, variable structure systems. I.
Nonlinear Adaptive Prediction of Speech with a Pipelined Recurrent Neural Network and Advanced Learning Algorithms
"... New learning algorithms for an adaptive nonlinear forward predictor which is based on a Pipelined Recurrent Neural Network (PRNN) are presented. A computationally efficient Gradient Descent (GD) algorithm, as well as a novel Extended Recursive Least Squares (ERLS) algorithm are tested on the pre ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
New learning algorithms for an adaptive nonlinear forward predictor which is based on a Pipelined Recurrent Neural Network (PRNN) are presented. A computationally efficient Gradient Descent (GD) algorithm, as well as a novel Extended Recursive Least Squares (ERLS) algorithm are tested on the predictor. Simulation studies, based on three speech signals, which have been made public and are available on the World Wide Web (WWW), show that the nonlinear predictor does not perform satisfactorily when the previously proposed gradient descent algorithm was used. The steepest descent algorithm is shown to yield a poor performance in terms of the prediction error gain, whereas consistently improved results are obtained using the ERLS algorithm. The merit of the nonlinear predictor structure is confirmed by yielding approximately 2 dB higher prediction gain than only a linear structure predictor, which uses the conventional Recursive Least Squares (RLS) algorithm. 1.1 Introduction...
Nonlinear predictive control based on neural multimodels
 International Journal of Applied Mathematics and Computer Science 20(1): 7–21, DOI
, 2010
"... This paper discusses neural multimodels based on Multi Layer Perceptron (MLP) networks and a computationally efficient nonlinear Model Predictive Control (MPC) algorithm which uses such models. Thanks to the nature of the model it calculates future predictions without using previous predictions. Th ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
This paper discusses neural multimodels based on Multi Layer Perceptron (MLP) networks and a computationally efficient nonlinear Model Predictive Control (MPC) algorithm which uses such models. Thanks to the nature of the model it calculates future predictions without using previous predictions. This means that, unlike the classical Nonlinear Auto Regressive with eXternal input (NARX) model, the multimodel is not used recurrently in MPC, and the prediction error is not propagated. In order to avoid nonlinear optimisation, in the discussed suboptimal MPC algorithm the neural multimodel is linearised online and, as a result, the future control policy is found by solving of a quadratic programming problem.
Oblique Support Vector Machines ∗
, 2005
"... Abstract. In this paper we propose a modified framework of support vector machines, called Oblique Support Vector Machines(OSVMs), to improve the capability of classification. The principle of OSVMs is joining an orthogonal vector into weight vector in order to rotate the support hyperplanes. By th ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract. In this paper we propose a modified framework of support vector machines, called Oblique Support Vector Machines(OSVMs), to improve the capability of classification. The principle of OSVMs is joining an orthogonal vector into weight vector in order to rotate the support hyperplanes. By this way, not only the regularized risk function is revised, but the constrained functions are also modified. Under this modification, the separating hyperplane and the margin of separation are constructed more precise. Moreover, in order to apply to largescale data problem, an iterative learning algorithm is proposed. In this iterative learning algorithm, three different schemes for training can be found in this literature, including patternmode learning, semibatch mode learning and batch mode learning. Besides, smooth technique is adopted in order to convert the constrained nonlinear programming problem into unconstrained optimum problem. Consequently, experimental results and comparisons are given to demonstrate that the performance of OSVMs is better than that of SVMs and SSVMs.
Autonomic Adaptation in Distributed Systems and Networks
"... There is now considerable interest in industry and academia for “autonomic” or selfadaptive networks and distributed systems. Largescale applications, such as simulation, typically run on large distributed systems in which it is impossible to guarantee at each instant the full reliability and avai ..."
Abstract
 Add to MetaCart
There is now considerable interest in industry and academia for “autonomic” or selfadaptive networks and distributed systems. Largescale applications, such as simulation, typically run on large distributed systems in which it is impossible to guarantee at each instant the full reliability and availability of all processing nodes and of all communication links. Nevertheless such systems have to accomplish their mission and provide a high level of dependability and the best possible performance to critical applications. Selfadaptation will therefore be an intrinsic feature of large distributed systems and networks in order to move tasks and files in response to changes in subsystem dependability and system load, and to provide QualityofService and dependable connections in the presence of fluctuating workloads and unknown system behavior. In this paper we review our work on the design of adaptive online task management algorithms for distributed systems, and the control of cognitive packet networks (CPN) to offer user specified QoS.