Analysis of incomplete climate data: Estimation of mean values and covariance matrices and imputation of missing values

Analysis of incomplete climate data: Estimation of mean values and covariance matrices and imputation of missing values (2001)

Cached

Download Links

by Tapio Schneider

Citations:

20 - 2 self

Versions

Version History

Metadata Version 2

User correction supplied by SystemCorrections

Datum	Value	Source
TITLE	Analysis of incomplete climate data: Estimation of mean values and covariance matrices and imputation of missing values	user correction - Legacy Corrections
AUTHOR NAME	Tapio Schneider	SVM HeaderParse 0.1
AUTHOR AFFIL	Atmospheric and Oceanic Sciences Program, Princeton University, Princeton, New Jersey	SVM HeaderParse 0.2
ABSTRACT	Estimating the mean and the covariance matrix of an incomplete dataset and filling in missing values with imputed values is generally a nonlinear problem, which must be solved iteratively. The expectation maximization (EM) algorithm for Gaussian data, an iterative method both for the estimation of mean values and covariance matrices from incomplete datasets and for the imputation of missing values, is taken as the point of departure for the development of a regularized EM algorithm. In contrast to the conventional EM algorithm, the regularized EM algorithm is applicable to sets of climate data, in which the number of variables typically exceeds the sample size. The regularized EM algorithm is based on iterated analyses of linear regressions of variables with missing values on variables with available values, with regression coefficients estimated by ridge regression, a regularized regression method in which a continuous regularization parameter controls the filtering of the noise in the data. The regularization parameter is determined by generalized cross-validation, such as to minimize, approximately, the expected mean squared error of the imputed values. The regularized EM algorithm can estimate, and exploit for the imputation of missing values, both synchronic and diachronic covariance matrices, which may contain information on spatial covariability, stationary temporal covariability, or cyclostationary temporal covariability. A test of the regularized EM algorithm with simulated surface temperature data demonstrates that the algorithm is applicable to typical sets of climate data and that it leads to more accurate estimates of the missing values than a conventional non-iterative imputation technique.	user correction - Legacy Corrections
YEAR	2001	user correction - Legacy Corrections
CITATIONS	37 found	ParsCit 1.0