constraints

We present a simple scheme for on-line estimation of covariance parameters in statistical data assimilation systems. The scheme is based on a maximumlikelihoodapproach in which estimates are produced on the basis of a single batch of simultaneous observations. Single-sample covariance estimation is reasonable as long as the number of available observations exceeds the number of tunable parameters by two or three orders of magnitude. Not much is known at present about model error associated with actual forecast systems. Our scheme can be used to estimate some important statistical model error parameters such as regionally averaged variances or characteristic correlation length scales. The advantage of the single-sample approach is that it does not rely on any assumptions about the temporal behavior of the covariance parameters: time-dependent parameter estimates can be continuously adjusted on the basis of current observations. This is of practical importance since it is likely to be th...

The maximum-likelihood method for estimating observation and forecast error covariance parameters is described. The method is presented in general terms but with particular emphasis on practical aspects of implementation. Issues such as bias estimation and correction, parameter identifiability, estimation accuracy, and robustness of the method, are discussed in detail. The relationship between the maximum-likelihood method and Generalized Cross-Validation is briefly addressed. The method can be regarded as a generalization of the traditional procedure for estimating covariance parameters from station data. It does not involve any restrictions on the covariance models and can be used with data from moving observers, provided the parameters to be estimated are identifiable. Any available a priori information about the observation and forecast error distributions can be incorporated into the estimation procedure. Estimates of parameter accuracy due to sampling error are obtained as a by-p...

Using an information point of view, we discuss deterministic versus stochastic tools for residual generation and evaluation for fault detection and isolation (FDI) in linear time invariant (LTI) state-space systems. In both types of approaches to off-line FDI, residual generation can be viewed as the design of a linear transformation of a Gaussian vector (the finite-window input-adjusted observations) . Several statistical isolation methods are revisited, using both a linear transform formulation and the information content of the corresponding residuals. We formally state several multiple fault cases, with or without causality assumptions, and discuss an optimality criterion for the most general one. New information criteria are proposed for investigating the residual optimization problem.

Abstract—Hyperspectral sensors are passive sensors that simultaneously record images for hundreds of contiguous and narrowly spaced regions of the electromagnetic spectrum. Each image corresponds to the same ground scene, thus creating a cube of images that contain both spatial and spectral information about the objects and backgrounds in the scene. In this paper, we present an adaptive anomaly detector designed assuming that the background clutter in the hyperspectral imagery is a three-dimensional Gauss–Markov random field. This model leads to an efficient and effective algorithm for discriminating man-made objects (the anomalies) in real hyperspectral imagery. The major focus of the paper is on the adaptive stage of the detector, i.e., the estimation of the Gauss–Markov random field parameters. We develop three methods: maximum-likelihood; least squares; and approximate maximum-likelihood. We study these approaches along three directions: estimation error performance, computational cost, and detection performance. In terms of estimation error, we derive the Cramér–Rao bounds and carry out Monte Carlo simulation studies that show that the three estimation procedures have similar performance when the fields are highly correlated, as is often the case with real hyperspectral imagery. The approximate maximum-likelihood method has a clear advantage from the computational point of view. Finally, we test extensively with real hyperspectral imagery the adaptive anomaly detector incorporating either the least squares or the approximate maximum-likelihood estimators. Its performance compares very favorably with that of the RX algorithm, an alternative detector commonly used with multispectral data, while reducing by up to an order of magnitude the associated computational cost. Index Terms—Anomaly detection, Cramér–Rao bounds, Gauss– Markov random field, hyperspectral imagery, least squares, maximum

By invoking the extended invariance principle (EXIP), we present herein a computationally efficient method that provides asymptotic (for large samples) maximum likelihood (AML) estimation for structured covariance matrices and will be referred to as the AML algorithm. A closed-form formula for estimating Hermitian Toeplitz covariance matrices that makes AML computationally simpler than most existing Hermitian Toeplitz matrix estimation algorithms is derived. Although the AML covariance matrix estimator can be used in a variety of applications, we focus on array processing in this paper. Our simulation study shows that AML enhances the performances of angle estimation algorithms, such as MUSIC, by making them very close to the corresponding Cram er--Rao bound (CRB) for uncorrelated signals. Numerical comparisons with several structured and unstructured covariance matrix estimators are also presented.

In this paper, a space-alternating generalized expectation-maximization (SAGE) algorithm is presented for the numerical computation of maximum-likelihood (ML) and penalized maximumlikelihood (PML) estimates of the parameters of covariance matrices with linear structure for complex Gaussian processes. By using a less informative hidden-data space and a sequential parameter-update scheme, a SAGE-based algorithm is derived for which convergence of the likelihood is demonstrated to be significantly faster than that of an EM-based algorithm that has been previously proposed. In addition, the SAGE procedure is shown to easily accommodate penalty functions, and a SAGE-based algorithm is derived and demonstrated for forming PML estimates with a quadratic smoothness penalty.

We address the problems of modeling Doppler-shifted wide-band Gaussian random processes and of estimating the Doppler parameter from a finite series of discrete-time samples. Relations between the continuous-time process, the Doppler shift parameter, and the discrete-time process obtained by sampling are established. Approximate rational models are proposed. Various estimators are proposed for Doppler parameter when the second-order statistics of the original continuous-time random process are known. The Cramer-Rao bound is derived. The estimators are compared experimentally on synthetic Doppler-shifted data. We also hint at some extensions of the method to non-stationary processes and time-varying Doppler shifts. 1. Introduction The Doppler-shift effect is well-known for narrow-band signals emitted by moving sources (Fig. 1). In that case, freshman's physics tells us that a harmonic wave of frequency ! 0 emitted by a point source moving toward a fixed receiver with speed v is observ...

In this paper, we aim at finding a nearest correlation matrix to a given symmetric matrix, measured by the componentwise weighted Frobenius norm, with a prescribed rank and bound constraints on its correlations. This is in general a non-convex and difficult problem due to the presence of the rank constraint. To deal with this difficulty, we first consider a penalized version of this problem and then apply the essential ideas of the majorization method to the penalized problem by solving iteratively a sequence of least squares correlation matrix problems without the rank constraint. The latter problems can be solved by a recently developed quadratically convergent smoothing Newton-BiCGStab method. Numerical examples demonstrate that our approach is very efficient for obtaining a nearest correlation matrix with both rank and bound constraints. Key words: correlation matrix, penalty method, majorization, least squares, Newton’s method 1

Image formation in radio astronomy is often posed as a problem of constructing a nonnegative function from sparse samples of its Fourier transform. We explore an alternative approach which reformulates the problem in terms of estimating the entries of a diagonal covariance matrix from Gaussian data. Maximum-likelihood estimates of the covariance cannot be readily computed analytically; hence we investigate an iterative algorithm originally proposed by Snyder, O'Sullivan, and Miller in the context of radar imaging. The resulting maximum-likelihood estimates tend to be unacceptably rough due to the ill-posed nature of maximum-likelihood estimation of functions from limited data, so some kind of regularization is needed. We explore penalized likelihoods based on entropy functionals, a roughness penalty proposed by Silverman, and an information-theoretic formulation of Good's roughness penalty crafted by O'Sullivan. We also investigate algorithm variations that perform a gene...