Results 1  10
of
18
On Tikhonov Regularization, Bias and Variance in Nonlinear System Identification
 Automatica
, 1997
"... this paper we study Tikhonov regularization (Tikhonov and Arsenin 1977). While Tikhonov regularization has had significant impact on several branches of science and engineering dealing with illposed and inverse problems, in particular modeling and analysis of highdimensional or distributed signals ..."
Abstract

Cited by 24 (6 self)
 Add to MetaCart
this paper we study Tikhonov regularization (Tikhonov and Arsenin 1977). While Tikhonov regularization has had significant impact on several branches of science and engineering dealing with illposed and inverse problems, in particular modeling and analysis of highdimensional or distributed signals and data (Tikhonov and Arsenin 1977, O'Sullivan 1986, Wahba 1990, Poggio
The Fundamental Role of General Orthonormal Bases in System Identification
 IEEE Transactions on Automatic Control
, 1997
"... The purpose of this paper is threefold. Firstly, it is to establish that contrary to what might be expected, the accuracy of well known and frequently used asymptotic variance results can depend on choices of fixed poles or zeros in the model structure. Secondly, it is to derive new variance express ..."
Abstract

Cited by 14 (10 self)
 Add to MetaCart
The purpose of this paper is threefold. Firstly, it is to establish that contrary to what might be expected, the accuracy of well known and frequently used asymptotic variance results can depend on choices of fixed poles or zeros in the model structure. Secondly, it is to derive new variance expressions that can provide greatly improved accuracy while also making explicit the influence of any fixed poles or zeros. This is achieved by employing certain new results on generalised Fourier series and the asymptotic properties of Toeplitzlike matrices in such a way that the new variance expressions presented here encompass preexisting ones as special cases. Via this latter analysis a new perspective emerges on recent work pertaining to the use of orthonormal basis structures in system identification. Namely, that orthonormal bases are much more than an implementational option offering improved numerical properties. In fact, they are an intrinsic part of estimation since, as shown here, or...
A Bootstrap Variant of AIC for StateSpace Model Selection
 STATISTICA SINICA
, 1997
"... Following in the recent work of Hurvich and Tsai (1989, 1991, 1993) and Hurvich, Shumway, and Tsai (1990), we propose a corrected variant of AIC developed for the purpose of smallsample statespace model selection. Our variant of AIC utilizes bootstrapping in the statespace framework (Stoffer and ..."
Abstract

Cited by 9 (4 self)
 Add to MetaCart
Following in the recent work of Hurvich and Tsai (1989, 1991, 1993) and Hurvich, Shumway, and Tsai (1990), we propose a corrected variant of AIC developed for the purpose of smallsample statespace model selection. Our variant of AIC utilizes bootstrapping in the statespace framework (Stoffer and Wall (1991)) to provide an estimate of the expected KullbackLeibler discrepancy between the model generating the data and a fitted approximating model. We present simulation results which demonstrate that in smallsample settings, our criterion estimates the expected discrepancy with less bias than traditional AIC and certain other competitors. As a result, our AIC variant serves as an effective tool for selecting a model of appropriate dimension. We present an asymptotic justification for our criterion in the Appendix.
Variance Error Quantifications that are Exact for Finite Model Order
 IEEE Transactions on Automatic Control
, 2003
"... This paper is concerned with the frequency domain quantification of noise induced errors in dynamic system estimates. Preceding seminal work on this problem provides general expressions that are approximations whose accuracy increases with observed data length and model order. In the interests of ..."
Abstract

Cited by 9 (5 self)
 Add to MetaCart
This paper is concerned with the frequency domain quantification of noise induced errors in dynamic system estimates. Preceding seminal work on this problem provides general expressions that are approximations whose accuracy increases with observed data length and model order. In the interests of improved accuracy, this paper provides new expressions whose accuracy depends only on data length.
Generalizing The Derivation Of The Schwarz Information Criterion
, 1999
"... The Schwarz information criterion (SIC, BIC, SBC) is one of the most widely known and used tools in statistical model selection. The criterion was derived by Schwarz (1978) to serve as an asymptotic approximation to a transformation of the Bayesian posterior probability of a candidate model. Althoug ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
The Schwarz information criterion (SIC, BIC, SBC) is one of the most widely known and used tools in statistical model selection. The criterion was derived by Schwarz (1978) to serve as an asymptotic approximation to a transformation of the Bayesian posterior probability of a candidate model. Although the original derivation assumes that the observed data is independent, identically distributed, and arising from a probability distribution in the regular exponential family, SIC has traditionally been used in a much larger scope of model selection problems. To better justify the widespread applicability of SIC, we derive the criterion in a very general framework: one which does not assume any specific form for the likelihood function, but only requires that it satisfies certain nonrestrictive regularity conditions.
Quantifying the Accuracy of Hammerstein Model Estimation
, 1999
"... This paper investigates the accuracy of the linear component that forms part of an overall Hammerstein modelstructure estimate, and a key finding is that the process of estimating the nonlinear element can have a strong effect on the associated estimate of the linear dynamics. Furthermore, this ef ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
This paper investigates the accuracy of the linear component that forms part of an overall Hammerstein modelstructure estimate, and a key finding is that the process of estimating the nonlinear element can have a strong effect on the associated estimate of the linear dynamics. Furthermore, this effect is not explained simply by way of considering how the input spectrum is changed by the nonlinearity. Instead, it arises that the linear modelestimate variability may be dominated by a term that depends on the frequency response of the linear system itself. Amongst other things, the main results derived here have experiment design implications for Hammerstein system estimation. Technical Report EE9933, Department of Electrical and Computer Engineering, University of Newcastle, AUSTRALIA 1
The Asymptotic CRLB for the Spectrum of ARMA Processes
 IEEE Transactions on Signal Processing
, 2003
"... This paper addresses the issue of quantifying the frequency domain accuracy of ARMA spectral estimates as dictated by the CramerRao Lower Bound (CRLB). Classical work in this area has led to expressions that are asymptotically exact as both data length and model order tend to infinity, although ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
This paper addresses the issue of quantifying the frequency domain accuracy of ARMA spectral estimates as dictated by the CramerRao Lower Bound (CRLB). Classical work in this area has led to expressions that are asymptotically exact as both data length and model order tend to infinity, although they are commonly used in finite model order and finite data length settings as approximations. More recent work has established quantifications which, for AR models, are exact for finite model order. By employing new analysis methods based on rational orthonormal parameterisations, together with the ideas of reproducing kernel Hilbert spaces, this paper develops quantifications that extend this previous work by being exact for finite model order in all of the AR, MA and ARMA system cases. These quantifications, via their explicit dependence on poles and zeros of the underlying spectral factor, reveal certain fundamental aspects of the accuracy achievable by spectral estimates of ARMA processes.
Information and Posterior Probability Criteria for Model Selection in Local Likelihood Estimation
 J Amer. Stat. Ass
, 1998
"... this paper we propose a modification to the methods used to motivate many information and posterior probability criteria for the weighted likelihood case. We derive weighted versions for two of the most widely known criteria, namely the AIC and BIC. Via a simple modification, the criteria are also m ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
this paper we propose a modification to the methods used to motivate many information and posterior probability criteria for the weighted likelihood case. We derive weighted versions for two of the most widely known criteria, namely the AIC and BIC. Via a simple modification, the criteria are also made useful for window span selection. The usefulness of the weighted version of these criteria are demonstrated through a simulation study and an application to three data sets. KEY WORDS: Information Criteria; Posterior Probability Criteria; Model Selection; Local Likelihood. 1. INTRODUCTION Local regression has become a popular method for smoothing scatterplots and for nonparametric regression in general. It has proven to be a useful tool in finding structure in datasets (Cleveland and Devlin 1988). Local regression estimation is a method for smoothing scatterplots (x i ; y i ), i = 1; : : : ; n in which the fitted value at x 0 is the value of a polynomial fit to the data using weighted least squares where the weight given to (x i ; y i ) is related to the distance between x i and x 0 . Stone (1977) shows that estimates obtained using the local regression methods have desirable theoretical properties. Recently, Fan (1993) has studied minimax properties of local linear regression. Tibshirani and Hastie (1987) extend the ideas of local regression to a local likelihood procedure. This procedure is designed for nonparametric regression modeling in situations where weighted least squares is inappropriate as an estimation method, for example binary data. Local regression may be viewed as a special case of local likelihood estimation. Tibshirani and Hastie (1987), Staniswalis (1989), and Loader (1999) apply local likelihood estimation to several types of data where local regressio...
Asymptotic properties of LeastSquares Estimates of HammersteinWiener Models
, 1999
"... This paper investigates the asymptotic properties of least squares estimates of HammersteinWiener model structures, and in doing so establishes consistency and asymptotic normality under fairly mild conditions on the additive noise process, the inputs and the static nonlinearities. In relation ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
This paper investigates the asymptotic properties of least squares estimates of HammersteinWiener model structures, and in doing so establishes consistency and asymptotic normality under fairly mild conditions on the additive noise process, the inputs and the static nonlinearities. In relation
Robust Frequency Response Estimation Accounting for Noise and Undermodelling
 in Proceedings of the American Control Conference
, 1992
"... This paper addresses the problem of providing bounds on estimated plant frequency responses in a form suitable for robust control design. Our approach is to consider the undermodelling as a particular realisation of a random variable and to derive bounds based on averages over all possible noise rea ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
This paper addresses the problem of providing bounds on estimated plant frequency responses in a form suitable for robust control design. Our approach is to consider the undermodelling as a particular realisation of a random variable and to derive bounds based on averages over all possible noise realisations and over all possible undermodelling realisations. We critically examine the performance of these bounds relative to those that would be obtained by fitting a high order model to the data and then truncating to a low order model. We also show that the parameters in the distribution for the undermodelling can be estimated from the data analagously to the way measurement noise variance is estimated from prediction errors. We propose several new estimators and examine their finite data and asymptotic properties. 1 Introduction It is usually assumed in robust control design that a nominal model is available with appropriate uncertainty bounds. This paper addresses the issue of how th...