Results 1  10
of
22
The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices
 Advances in Mathematics 227 (2011
"... Abstract. In this paper, we consider the eigenvalues and eigenvectors of finite, low rank perturbations of random matrices. Specifically, we prove almost sure convergence of the extreme eigenvalues and appropriate projections of the corresponding eigenvectors of the perturbed matrix for additive and ..."
Abstract

Cited by 15 (4 self)
 Add to MetaCart
Abstract. In this paper, we consider the eigenvalues and eigenvectors of finite, low rank perturbations of random matrices. Specifically, we prove almost sure convergence of the extreme eigenvalues and appropriate projections of the corresponding eigenvectors of the perturbed matrix for additive and multiplicative perturbation models. The limiting nonrandom value is shown to depend explicitly on the limiting spectral measure and the assumed perturbation model via integral transforms that correspond to very well known objects in free probability theory that linearize noncommutative free additive and multiplicative convolution. Moreover, we uncover a remarkable phase transition phenomenon whereby the large matrix limit of the extreme eigenvalues of the perturbed matrix differs from that of the original matrix if and only if the eigenvalues of the perturbing matrix are above a certain critical threshold. This critical threshold is intimately related to the same aforementioned integral transforms. We examine the consequence of this eigenvalue phase transition on the associated eigenvectors and generalize our results to examine the singular values and vectors of finite, low rank perturbations of rectangular random matrices. The analysis brings into sharp focus the analogous connection with rectangular free probability. Various extensions of our results are discussed. 1.
Treelets — An Adaptive MultiScale Basis for Sparse Unordered Data
"... In many modern applications, including analysis of gene expression and text documents, the data are noisy, highdimensional, and unordered — with no particular meaning to the given order of the variables. Yet, successful learning is often possible due to sparsity: the fact that the data are typicall ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
In many modern applications, including analysis of gene expression and text documents, the data are noisy, highdimensional, and unordered — with no particular meaning to the given order of the variables. Yet, successful learning is often possible due to sparsity: the fact that the data are typically redundant with underlying structures that can be represented by only a few features. In this paper, we present treelets — a novel construction of multiscale bases that extends wavelets to nonsmooth signals. The method is fully adaptive, as it returns a hierarchical tree and an orthonormal basis which both reflect the internal structure of the data. Treelets are especially wellsuited as a dimensionality reduction and feature selection tool prior to regression and classification, in situations where sample sizes are small and the data are sparse with unknown groupings of correlated or collinear variables. The method is also simple to implement and analyze theoretically. Here we describe a variety of situations where treelets perform better than principal component analysis as well as some common variable selection and cluster averaging schemes. We illustrate treelets on a blocked covariance model and on several data sets (hyperspectral image data, DNA microarray data, and internet advertisements) with highly complex dependencies between variables. 1
TREELETS—AN ADAPTIVE MULTISCALE BASIS FOR SPARSE UNORDERED DATA
"... In many modern applications, including analysis of gene expression and text documents, the data are noisy, highdimensional, and unordered—with no particular meaning to the given order of the variables. Yet, successful learning is often possible due to sparsity: the fact that the data are typically ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
In many modern applications, including analysis of gene expression and text documents, the data are noisy, highdimensional, and unordered—with no particular meaning to the given order of the variables. Yet, successful learning is often possible due to sparsity: the fact that the data are typically redundant with underlying structures that can be represented by only a few features. In this paper we present treelets—a novel construction of multiscale bases that extends wavelets to nonsmooth signals. The method is fully adaptive, as it returns a hierarchical tree and an orthonormal basis which both reflect the internal structure of the data. Treelets are especially wellsuited as a dimensionality reduction and feature selection tool prior to regression and classification, in situations where sample sizes are small and the data are sparse with unknown groupings of correlated or collinear variables. The method is also simple to implement and analyze theoretically. Here we describe a variety of situations where treelets perform better than principal component analysis, as well as some common variable selection and cluster averaging schemes. We illustrate treelets on a blocked covariance model and on several data sets (hyperspectral image data, DNA microarray data, and internet advertisements) with highly complex dependencies between variables.
REGRESSION ON MANIFOLDS: ESTIMATION OF THE EXTERIOR DERIVATIVE
 SUBMITTED TO THE ANNALS OF STATISTICS
, 2010
"... Collinearity and nearcollinearity of predictors cause difficulties when doing regression. In these cases, variable selection becomes untenable because of mathematical issues concerning the existence and numerical stability of the regression coefficients, and interpretation of the coefficients is am ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
Collinearity and nearcollinearity of predictors cause difficulties when doing regression. In these cases, variable selection becomes untenable because of mathematical issues concerning the existence and numerical stability of the regression coefficients, and interpretation of the coefficients is ambiguous because gradients are not defined. Using a differential geometric interpretation, in which the regression coefficients are interpreted as estimates of the exterior derivative of a function, we develop a new method to do regression in the presence of collinearities. Our regularization scheme can improve estimation error, and it can be easily modified to include lassotype regularization. These estimators also have simple extensions to the “large p, small n” context.
NonParametric Detection of Signals by Information Theoretic Criteria: Performance Analysis and an Improved Estimator
, 2009
"... Determining the number of sources is a fundamental problem in many scientific fields. In this paper we consider the nonparametric setting, and focus on the detection performance of two popular estimators based on information theoretic criteria, the Akaike information criterion (AIC) and minimum des ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
Determining the number of sources is a fundamental problem in many scientific fields. In this paper we consider the nonparametric setting, and focus on the detection performance of two popular estimators based on information theoretic criteria, the Akaike information criterion (AIC) and minimum description length (MDL). We present three contributions on this subject. First, we derive a new expression for the detection performance of the MDL estimator, which exhibits a much closer fit to simulations in comparison to previous formulas. Second, we present a random matrix theory viewpoint of the performance of the AIC estimator, including approximate analytical formulas for its overestimation probability. Finally, we show that a small increase in the penalty term of AIC leads to an estimator with a very good detection performance and a negligible overestimation probability.
The singular values and vectors of low rank perturbations of large rectangular random matrices
 J. Multivariate Anal
"... Abstract. In this paper, we consider the singular values and singular vectors of finite, low rank perturbations of large rectangular random matrices. Specifically, we prove almost sure convergence of the extreme singular values and appropriate projections of the corresponding singular vectors of the ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
Abstract. In this paper, we consider the singular values and singular vectors of finite, low rank perturbations of large rectangular random matrices. Specifically, we prove almost sure convergence of the extreme singular values and appropriate projections of the corresponding singular vectors of the perturbed matrix. As in the prequel, where we considered the eigenvalues of Hermitian matrices, the nonrandom limiting value is shown to depend explicitly on the limiting singular value distribution of the unperturbed matrix via an integral transform that linearizes rectangular additive convolution in free probability theory. The asymptotic position of the extreme singular values of the perturbed matrix differs from that of the original matrix if and only if the singular values of the perturbing matrix are above a certain critical threshold which depends on this same aforementioned integral transform. We examine the consequence of this singular value phase transition on the associated left and rightsingulareigenvectorsand discuss the fluctuations aroundthese nonrandom limits. 1.
Minimax rates of estimation for sparse PCA in high dimensions
, 2012
"... We study sparse principal components analysis in the highdimensional setting, where p (the number of variables) can be much larger than n (the number of observations). We prove optimal, nonasymptotic lower and upper bounds on the minimax estimation error for the leading eigenvector when it belongs ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
We study sparse principal components analysis in the highdimensional setting, where p (the number of variables) can be much larger than n (the number of observations). We prove optimal, nonasymptotic lower and upper bounds on the minimax estimation error for the leading eigenvector when it belongs to an ℓq ball for q ∈ [0, 1]. Our bounds are sharp in p and n for all q ∈ [0, 1] over a wide class of distributions. The upper bound is obtained by analyzing the performance of ℓqconstrained PCA. In particular, our results provide convergence rates for ℓ1constrained PCA. 1
Refined Perturbation Bounds for Eigenvalues of Hermitian and NonHermitian Matrices, to appear
 SIAM J. Matrix Analysis
, 2008
"... Abstract. We present eigenvalue bounds for perturbations of Hermitian matrices, and express the change in eigenvalues in terms of a projection of the perturbation onto a particular eigenspace, rather than in terms of the full perturbation. The perturbations we consider are Hermitian of rank one, and ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
Abstract. We present eigenvalue bounds for perturbations of Hermitian matrices, and express the change in eigenvalues in terms of a projection of the perturbation onto a particular eigenspace, rather than in terms of the full perturbation. The perturbations we consider are Hermitian of rank one, and Hermitian or nonHermitian with norm smaller than the spectral gap of a specific eigenvalue. Applications include principal component analysis under a spiked covariance model, and pseudo arclength continuation methods for the solution of nonlinear systems.
Recursive sparse recovery in large but structured noise  part 2,” arXiv: 1211.3754 [cs.IT
, 2013
"... Abstract—We study the problem of recursively recovering a time sequence of sparse vectors, St, from measurements Mt: = St + Lt that are corrupted by structured noise Lt which is dense and can have large magnitude. The structure that we require is that Lt should lie in a low dimensional subspace that ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
Abstract—We study the problem of recursively recovering a time sequence of sparse vectors, St, from measurements Mt: = St + Lt that are corrupted by structured noise Lt which is dense and can have large magnitude. The structure that we require is that Lt should lie in a low dimensional subspace that is either fixed or changes “slowly enough”; and the eigenvalues of its covariance matrix are “clustered”. We do not assume any model on the sequence of sparse vectors. Their support sets and their nonzero element values may be either independent or correlated over time (usually in many applications they are correlated). The only thing required is that there be some support change every so often. We introduce a novel solution approach called Recursive Projected Compressive Sensing with clusterPCA (ReProCScPCA) that addresses some of the limitations of earlier work. Under mild assumptions, we show that, with high probability, ReProCScPCA can exactly recover the support set of St at all times; and the reconstruction errors of both St and Lt are upper bounded by a timeinvariant and small value. I.
Hypothesis Testing and Random Matrix Theory
"... Abstract—Detection of the number of signals embedded in noise is a fundamental problem in signal and array processing. This paper focuses on the nonparametric setting where no knowledge of the array manifold is assumed. First, we present a detailed statistical analysis of this problem, including an ..."
Abstract
 Add to MetaCart
Abstract—Detection of the number of signals embedded in noise is a fundamental problem in signal and array processing. This paper focuses on the nonparametric setting where no knowledge of the array manifold is assumed. First, we present a detailed statistical analysis of this problem, including an analysis of the signal strength required for detection with high probability, and the form of the optimal detection test under certain conditions where such a test exists. Second, combining this analysis with recent results from random matrix theory, we present a new algorithm for detection of the number of sources via a sequence of hypothesis tests. We theoretically analyze the consistency and detection performance of the proposed algorithm, showing its superiority compared to the standard minimum description length (MDL)based estimator. A series of simulations confirm our theoretical analysis. Index Terms—Detection, number of signals, random matrix theory, statistical hypothesis tests, Tracy–Widom distribution.