Results 1  10
of
15
Robust Full Bayesian Learning for Neural Networks
, 1999
"... In this paper, we propose a hierarchical full Bayesian model for neural networks. This model treats the model dimension (number of neurons), model parameters, regularisation parameters and noise parameters as random variables that need to be estimated. We develop a reversible jump Markov chain Monte ..."
Abstract

Cited by 12 (9 self)
 Add to MetaCart
In this paper, we propose a hierarchical full Bayesian model for neural networks. This model treats the model dimension (number of neurons), model parameters, regularisation parameters and noise parameters as random variables that need to be estimated. We develop a reversible jump Markov chain Monte Carlo (MCMC) method to perform the necessary computations. We find that the results obtained using this method are not only better than the ones reported previously, but also appear to be robust with respect to the prior specification. In addition, we propose a novel and computationally efficient reversible jump MCMC simulated annealing algorithm to optimise neural networks. This algorithm enables us to maximise the joint posterior distribution of the network parameters and the number of basis function. It performs a global search in the joint space of the parameters and number of parameters, thereby surmounting the problem of local minima. We show that by calibrating the full hierarchical ...
Nonparametric Identification of Wiener Systems by Orthogonal Series
, 1994
"... ... a linear dynamic and a nonlinear memoryless subsystems connected in a cascade, is identified. Both the input signal and disturbance are random, white, and Gaussian. The unknown nonlinear characteristic is strictly monotonous and differentiable and, therefore, the problem of its recovering from i ..."
Abstract

Cited by 12 (1 self)
 Add to MetaCart
... a linear dynamic and a nonlinear memoryless subsystems connected in a cascade, is identified. Both the input signal and disturbance are random, white, and Gaussian. The unknown nonlinear characteristic is strictly monotonous and differentiable and, therefore, the problem of its recovering from inputoutput observations of the whole system is nonparametric. It is shown that the inverse of the characteristic is a regression function and, next, a class of orthogonal series nonparametric estimates recovering the regression is proposed and analyzed. The estimates apply the trigonometric, Legendre, and Hermite orthogonal functions. Pointwise consistency of all the algorithms is shown. Under some additional smoothness restrictions, the rates of their convergence are examined and compared. An algorithm to identify the impulse response of the linear subsystem is proposed.
Bayesian Methods for Neural Networks
, 1999
"... Summary The application of the Bayesian learning paradigm to neural networks results in a flexible and powerful nonlinear modelling framework that can be used for regression, density estimation, prediction and classification. Within this framework, all sources of uncertainty are expressed and meas ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
Summary The application of the Bayesian learning paradigm to neural networks results in a flexible and powerful nonlinear modelling framework that can be used for regression, density estimation, prediction and classification. Within this framework, all sources of uncertainty are expressed and measured by probabilities. This formulation allows for a probabilistic treatment of our a priori knowledge, domain specific knowledge, model selection schemes, parameter estimation methods and noise estimation techniques. Many researchers have contributed towards the development of the Bayesian learning approach for neural networks. This thesis advances this research by proposing several novel extensions in the areas of sequential learning, model selection, optimisation and convergence assessment. The first contribution is a regularisation strategy for sequential learning based on extended Kalman filtering and noise estimation via evidence maximisation. Using the expectation maximisation (EM) algorithm, a similar algorithm is derived for batch learning. Much of the thesis is, however, devoted to Monte Carlo simulation methods. A robust Bayesian method is proposed to estimate,
Stochastic Approximation in Nonparametric Identification of Hammerstein Systems
, 2002
"... Derived from the idea of stochastic approximation, recursive algorithms to identify a Hammerstein system are presented. Two of them recover the characteristic of the nonlinear memoryless subsystem while the third one estimates the impulse response of the linear dynamic part. The a priori informatio ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
Derived from the idea of stochastic approximation, recursive algorithms to identify a Hammerstein system are presented. Two of them recover the characteristic of the nonlinear memoryless subsystem while the third one estimates the impulse response of the linear dynamic part. The a priori information about both subsystems is nonparametric. Consistency in quadratic mean is shown and the convergence rate is examined. Results of numerical simulation are also presented.
Identification of linear systems withnonlinear distortions
, 2003
"... In this paper the impact of nonlinear distortions on the linear system identification framework is studied. In the first part the nonlinear system is replaced by a linear model plus a nonlinear noise source. The properties of this representation are studied. Next a method to detect, qualify and qua ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
In this paper the impact of nonlinear distortions on the linear system identification framework is studied. In the first part the nonlinear system is replaced by a linear model plus a nonlinear noise source. The properties of this representation are studied. Next a method to detect, qualify and quantify the nonlinear distortions is presented. In the second part, the (non)parametric identification of the best linear approximation is studied. In the last part, the linear modelling approach is extended towards nonlinear modelling. A fast approximate nonlinear modelling framework is set up that is a natural extension of the linear framework, and bridges the gap between the linear and the nonlinear identification approaches.
Identification of Multivariable Hammerstein Systems using Rational Orthonormal Bases
"... In this paper, a non iterative algorithm for the simultaneous identification of the linear and nonlinear parts of multivariable Hammerstein systems is presented. The proposed algorithm is numerically robust, since it is based only on least squares estimation and singular value decomposition. Under w ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
In this paper, a non iterative algorithm for the simultaneous identification of the linear and nonlinear parts of multivariable Hammerstein systems is presented. The proposed algorithm is numerically robust, since it is based only on least squares estimation and singular value decomposition. Under weak assumptions on the persistency of excitation of the inputs, the algorithm provides consistent estimates even in the presence of coloured noise. Key in the derivation of the results is the use of rational orthonormal bases for the representation of the linear part of the system. An additional advantage of this is the possibility of incorporating prior information about the system in a typically blackbox identification scheme. 1 Introduction In the last decades, many research activities have been carried out on modelling, identification, and control design of nonlinear systems. Many dynamical systems can be better represented by nonlinear models, which are able to describe the global be...
Identifying nonlinear wave interactions in plasmas using twopoint measurements: a case study of SLAMS
, 1998
"... . Two fundamental quantities for characterizing nonlinear wave phenomena in plasmas are the spectral energy transfer associated with the energy redistribution between Fourier modes, and the linear growth rate. It is shown how these quantities can be estimated simultaneously from dual spacecraft data ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
. Two fundamental quantities for characterizing nonlinear wave phenomena in plasmas are the spectral energy transfer associated with the energy redistribution between Fourier modes, and the linear growth rate. It is shown how these quantities can be estimated simultaneously from dual spacecraft data using Volterra series models. We consider magnetic field data gathered upstream the Earth's quasiparallel bow shock, in which Short Large Amplitude Magnetic Structures (SLAMS) supposedly play a leading role. The analysis attests the dynamic evolution of the SLAMS and reveals an energy cascade toward high frequency waves. These results put constraints on possible mechanisms for the shock front formation. 3 1. Introduction Our lack of understanding on plasma turbulence in the last decades has spurred an intensive quest for alternative ways of characterizing nonlinear processes. In this paper, we use a particular representation, based on Volterra series, to describe the evolution in time an...
Maximum Likelihood Identification of Wiener Models
"... The Wiener model is a block oriented model having a linear dynamic system followed by a static nonlinearity. The dominating approach to estimate the components of this model has been to minimize the error between the simulated and the measured outputs. We show that this will in general lead to bia ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
The Wiener model is a block oriented model having a linear dynamic system followed by a static nonlinearity. The dominating approach to estimate the components of this model has been to minimize the error between the simulated and the measured outputs. We show that this will in general lead to biased estimates if there is other disturbances present than measurement noise. The implications of Bussgang’s theorem in this context are also discussed. For the case with general disturbances we derive the Maximum Likelihood method and show how it can be efficiently implemented. Comparisons between this new algorithm and the traditional approach confirm that the new method is unbiased and also has superior accuracy.
Identification of Nonlinear Systems using Orthonormal Bases
, 2001
"... In this paper, non iterative algorithms for the identification of (multivariable) nonlinear systems consisting of the interconnection of LTI systems and static nonlinearities are presented. The proposed algorithms are numerically robust, since they are based only on least squares estimation and sing ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
In this paper, non iterative algorithms for the identification of (multivariable) nonlinear systems consisting of the interconnection of LTI systems and static nonlinearities are presented. The proposed algorithms are numerically robust, since they are based only on least squares estimation and singular value decomposition. Three di#erent blockoriented nonlinear models are considered in this paper, viz., the Hammerstein model, the Wiener model, and the Feedback BlockOriented model. For the Hammerstein model, the algorithm provides consistent estimates even in the presence of coloured output noise, under weak assumptions on the persistency of excitation of the inputs. For the Wiener model and the Feedback BlockOriented model, consistency of the estimates can only be guaranteed in the noise free case. Key in the derivation of the results is the use of rational orthonormal bases for the representation of the linear part of the systems. An additional advantage of this is the possibility of incorporating prior information about the system in a typically blackbox identification scheme.
IMTC 2003  Instrumentation and Measurement
, 2003
"... Nowadays for solving fault diagnosis, fault isolation, and control problems a possible solution can be the use of modelbased approaches, which contain a representation of our knowledge about the faultfree and faulty systems. If the nature and/or the actual circumstances of the problem in hand chang ..."
Abstract
 Add to MetaCart
Nowadays for solving fault diagnosis, fault isolation, and control problems a possible solution can be the use of modelbased approaches, which contain a representation of our knowledge about the faultfree and faulty systems. If the nature and/or the actual circumstances of the problem in hand change the corresponding model should also be changed. Anytime techniques are very flexible in this respect and can advantageously be used when the operation should be performed under changing circumstances. This paper provides the possible anytime extension of the modeling issues of TakagiSugeno fuzzy inference operator based approximation which is the essential core of the parallel distributed compensation (PDC) design approach. The PDC based system analysis and the fuzzy observer design are focused on a benchmark problem of fault diagnosis of an actuator in a sugar factory.