Results 1  10
of
23
Covariance regularization by thresholding
, 2007
"... This paper considers regularizing a covariance matrix of p variables estimated from n observations, by hard thresholding. We show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense, the variables are Gaussian or subGa ..."
Abstract

Cited by 69 (9 self)
 Add to MetaCart
This paper considers regularizing a covariance matrix of p variables estimated from n observations, by hard thresholding. We show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense, the variables are Gaussian or subGaussian, and (log p)/n → 0, and obtain explicit rates. The results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity. We discuss an intuitive resampling scheme for threshold selection and prove a general crossvalidation result that justifies this approach. We also compare thresholding to other covariance estimators in simulations and on an example from climate data. 1. Introduction. Estimation
Operator norm consistent estimation of largedimensional sparse covariance matrices
 Annals of Statistics
"... Estimating covariance matrices is a problem of fundamental importance in multivariate statistics. In practice it is increasingly frequent to work with data matrices X of dimension n×p, where p and n are both large. Results from random matrix theory show very clearly that in this setting, standard es ..."
Abstract

Cited by 22 (0 self)
 Add to MetaCart
Estimating covariance matrices is a problem of fundamental importance in multivariate statistics. In practice it is increasingly frequent to work with data matrices X of dimension n×p, where p and n are both large. Results from random matrix theory show very clearly that in this setting, standard estimators like the sample covariance matrix perform in general very poorly. In this “large n, large p ” setting, it is sometimes the case that practitioners are willing to assume that many elements of the population covariance matrix are equal to 0, and hence this matrix is sparse. We develop an estimator to handle this situation. The estimator is shown to be consistent in operator norm, when, for instance, we have p ≍ n as n → ∞. In other words the largest singular value of the difference between the estimator and the population covariance matrix goes to zero. This implies consistency of all the eigenvalues and consistency of eigenspaces associated to isolated eigenvalues. We also propose a notion of sparsity for matrices, that is, “compatible” with spectral analysis and is independent of the ordering of the variables. 1. Introduction. Estimating
Statistical eigeninference from large Wishart matrices
 Annals of Statistics
, 2008
"... We consider settings where the observations are drawn from a zeromean multivariate (real or complex) normal distribution with the population covariance matrix having eigenvalues of arbitrary multiplicity. We assume that the eigenvectors of the population covariance matrix are unknown and focus on i ..."
Abstract

Cited by 13 (3 self)
 Add to MetaCart
We consider settings where the observations are drawn from a zeromean multivariate (real or complex) normal distribution with the population covariance matrix having eigenvalues of arbitrary multiplicity. We assume that the eigenvectors of the population covariance matrix are unknown and focus on inferential procedures that are based on the sample eigenvalues alone (i.e., “eigeninference”). Results found in the literature establish the asymptotic normality of the fluctuation in the trace of powers of the sample covariance matrix. We develop concrete algorithms for analytically computing the limiting quantities and the covariance of the fluctuations. We exploit the asymptotic normality of the trace of powers of the sample covariance matrix to develop eigenvaluebased procedures for testing and estimation. Specifically, we formulate a simple test of hypotheses for the population eigenvalues and a technique for estimating the population eigenvalues in settings where the cumulative distribution function of the (nonrandom) population eigenvalues has a staircase structure. Monte Carlo simulations are used to demonstrate the superiority of the proposed
EigenInference for Energy Estimation of Multiple Sources
"... This paper introduces a new method to blindly estimate the transmit power of multiple signal sources in multiantenna fading channels, when the number of sensing devices and the number of available samples are sufficiently large compared to the number of sources. This work makes use of recent advanc ..."
Abstract

Cited by 9 (6 self)
 Add to MetaCart
This paper introduces a new method to blindly estimate the transmit power of multiple signal sources in multiantenna fading channels, when the number of sensing devices and the number of available samples are sufficiently large compared to the number of sources. This work makes use of recent advances in the field of large dimensional random matrix theory that result in a simple and computationally efficient consistent estimator of the power of each source. We provide a criterion to determine the minimum number of sensors and the minimum number of samples required to achieve source separation. Simulations are performed that corroborate the theoretical claims and show that the proposed power estimator largely outperforms alternative power inference techniques.
Concentration of measure and spectra of random matrices: with applications to correlation matrices, elliptical distributions and beyond
 THE ANNALS OF APPLIED PROBABILITY TO APPEAR
, 2009
"... We place ourselves in the setting of highdimensional statistical inference, where the number of variables p in a dataset of interest is of the same order of magnitude as the number of observations n. More formally we study the asymptotic properties of correlation and covariance matrices under the s ..."
Abstract

Cited by 8 (5 self)
 Add to MetaCart
We place ourselves in the setting of highdimensional statistical inference, where the number of variables p in a dataset of interest is of the same order of magnitude as the number of observations n. More formally we study the asymptotic properties of correlation and covariance matrices under the setting that p/n → ρ ∈ (0, ∞), for general population covariance. We show that spectral properties for large dimensional correlation matrices are similar to those of large dimensional covariance matrices, for a large class of models studied in random matrix theory. We also derive a MarčenkoPastur type system of equations for the limiting spectral distribution of covariance matrices computed from data with elliptical distributions and generalizations of this family. The motivation for this study comes partly from the possible relevance of such distributional assumptions to problems in econometrics and portfolio optimization, as well as robustness questions for certain classical random matrix results. A mathematical theme of the paper is the important use we make of concentration inequalities.
Resolvent of large random graphs
 Random Structures and Algorithms
"... We analyze the convergence of the spectrum of large random graphs to the spectrum of a limit infinite graph. We apply these results to graphs converging locally to trees and derive a new formula for the Stieltjes transform of the spectral measure of such graphs. We illustrate our results on the unif ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
We analyze the convergence of the spectrum of large random graphs to the spectrum of a limit infinite graph. We apply these results to graphs converging locally to trees and derive a new formula for the Stieltjes transform of the spectral measure of such graphs. We illustrate our results on the uniform regular graphs, ErdösRényi graphs and graphs with a given degree sequence. We give examples of application for weighted graphs, bipartite graphs and the uniform spanning tree of n vertices. MSCclass: 05C80, 15A52 (primary), 47A10 (secondary). 1
Highdimensionality effects in the Markowitz problem and other quadratic programs with linear equality constraints: risk underestimation
"... We study the properties of solutions of quadratic programs with linear equality constraints whose parameters are estimated from data in the highdimensional setting where p, the number of variables in the problem, is of the same order of magnitude as n, the number of observations used to estimate th ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
We study the properties of solutions of quadratic programs with linear equality constraints whose parameters are estimated from data in the highdimensional setting where p, the number of variables in the problem, is of the same order of magnitude as n, the number of observations used to estimate the parameters. The Markowitz problem in Finance is a subcase of our study. Assuming normality and independence of the observations we relate the efficient frontier computed empirically to the “true” efficient frontier. Our computations show that there is a separation of the errors induced by estimating the mean of the observations and estimating the covariance matrix. In particular, the price paid for estimating the covariance matrix is an underestimation of the variance by a factor roughly equal to 1 − p/n. Therefore the risk of the optimal population solution is underestimated when we estimate it by solving a similar quadratic program with estimated parameters. We also characterize the statistical behavior of linear functionals of the empirical optimal vector and show that they are biased estimators of the corresponding population quantities. We investigate the robustness of our Gaussian results by extending the study to certain elliptical models and models where our n observations are correlated (in “time”). We show a lack of robustness of the Gaussian results, but are still able to get results concerning first order properties of the quantities of interest, even in the case of relatively heavytailed data (we require two moments). Risk underestimation is still present in the elliptical case and more pronounced that in the Gaussian case. We discuss properties of the nonparametric and parametric bootstrap in this context. We show several results, including the interesting fact that standard applications of the bootstrap generally yields inconsistent estimates of bias. Finally, we propose some strategies to correct these problems and practically validate them in some simulations. In all the paper, we will assume that p, n and n − p tend to infinity, and p < n. 1
Approximating the Covariance Matrix with Lowrank Perturbations
"... Abstract. Covariance matrices capture correlations that are invaluable in modeling reallife datasets. Using all d 2 elements of the covariance (in d dimensions) is costly and could result in overfitting; and the simple diagonal approximation can be overrestrictive. We present an algorithm that im ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
Abstract. Covariance matrices capture correlations that are invaluable in modeling reallife datasets. Using all d 2 elements of the covariance (in d dimensions) is costly and could result in overfitting; and the simple diagonal approximation can be overrestrictive. We present an algorithm that improves upon the diagonal matrix by allowing a low rank perturbation. The efficiency is comparable to the diagonal approximation, yet one can capture correlations among the dimensions. We show that this method outperforms the diagonal when training GMMs on both synthetic and realworld data. Keywords: Gaussian Mixture models; efficient; maximum likelihood; EM 1
Sparsity and the Possibility of Inference
, 2008
"... We discuss the importance of sparsity in the context of nonparametric regression and covariance matrix estimation. We point to low manifold dimension of the covariate vector as a possible important feature of sparsity, recall an estimate of dimension due to Levina and Bickel (2005) and establish som ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
We discuss the importance of sparsity in the context of nonparametric regression and covariance matrix estimation. We point to low manifold dimension of the covariate vector as a possible important feature of sparsity, recall an estimate of dimension due to Levina and Bickel (2005) and establish some conjectures made in that paper. AMS (2000) subject classification.
Covariance Estimation: The GLM and Regularization Perspectives
"... Finding an unconstrained and statistically interpretable reparameterization of a covariance matrix is still an open problem in statistics. Its solution is of central importance in covariance estimation, particularly in the recent highdimensional data environment where enforcing the positivedefinit ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Finding an unconstrained and statistically interpretable reparameterization of a covariance matrix is still an open problem in statistics. Its solution is of central importance in covariance estimation, particularly in the recent highdimensional data environment where enforcing the positivedefiniteness constraint could be computationally expensive. We provide a survey of the progress made in modeling covariance matrices from the perspectives of generalized linear models (GLM) or parsimony and use of covariates in low dimensions, regularization (shrinkage, sparsity) for highdimensional data, and the role of various matrix factorizations. A viable and emerging regressionbased setup which is suitable for both the GLM and the regularization approaches is to link a covariance matrix, its inverse or their factors to certain regression models and then solve the relevant (penalized) least squares problems. We point out several instances of this regressionbased setup in the literature. A notable case is in the Gaussian graphical models where linear regressions with LASSO penalty are used to estimate the neighborhood of one node at a time (Meinshausen and Bühlmann, 2006). Some advantages