Results 1 
8 of
8
The spectrum of kernel random matrices
, 2007
"... We place ourselves in the setting of highdimensional statistical inference, where the number of variables p in a dataset of interest is of the same order of magnitude as the number of observations n. We consider the spectrum of certain kernel random matrices, in particular n × n matrices whose (i, ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
We place ourselves in the setting of highdimensional statistical inference, where the number of variables p in a dataset of interest is of the same order of magnitude as the number of observations n. We consider the spectrum of certain kernel random matrices, in particular n × n matrices whose (i, j)th entry is f(X ′ i Xj/p) or f(‖Xi − Xj ‖ 2 /p), where p is the dimension of the data, and Xi are independent data vectors. Here f is assumed to be a locally smooth function. The study is motivated by questions arising in statistics and computer science, where these matrices are used to perform, among other things, nonlinear versions of principal component analysis. Surprisingly, we show that in highdimensions, and for the models we analyze, the problem becomes essentially linear which is at odds with heuristics sometimes used to justify the usage of these methods. The analysis also highlights certain peculiarities of models widely studied in random matrix theory and raises some questions about their relevance as tools to model highdimensional data encountered in practice. 1
Highdimensionality effects in the Markowitz problem and other quadratic programs with linear equality constraints: risk underestimation
"... We study the properties of solutions of quadratic programs with linear equality constraints whose parameters are estimated from data in the highdimensional setting where p, the number of variables in the problem, is of the same order of magnitude as n, the number of observations used to estimate th ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
We study the properties of solutions of quadratic programs with linear equality constraints whose parameters are estimated from data in the highdimensional setting where p, the number of variables in the problem, is of the same order of magnitude as n, the number of observations used to estimate the parameters. The Markowitz problem in Finance is a subcase of our study. Assuming normality and independence of the observations we relate the efficient frontier computed empirically to the “true” efficient frontier. Our computations show that there is a separation of the errors induced by estimating the mean of the observations and estimating the covariance matrix. In particular, the price paid for estimating the covariance matrix is an underestimation of the variance by a factor roughly equal to 1 − p/n. Therefore the risk of the optimal population solution is underestimated when we estimate it by solving a similar quadratic program with estimated parameters. We also characterize the statistical behavior of linear functionals of the empirical optimal vector and show that they are biased estimators of the corresponding population quantities. We investigate the robustness of our Gaussian results by extending the study to certain elliptical models and models where our n observations are correlated (in “time”). We show a lack of robustness of the Gaussian results, but are still able to get results concerning first order properties of the quantities of interest, even in the case of relatively heavytailed data (we require two moments). Risk underestimation is still present in the elliptical case and more pronounced that in the Gaussian case. We discuss properties of the nonparametric and parametric bootstrap in this context. We show several results, including the interesting fact that standard applications of the bootstrap generally yields inconsistent estimates of bias. Finally, we propose some strategies to correct these problems and practically validate them in some simulations. In all the paper, we will assume that p, n and n − p tend to infinity, and p < n. 1
Effect of finite rate feedback on CDMA signature optimization and MIMO beamforming vector selection
 IEEE Trans. Inf. Theory
, 2009
"... Abstract—We analyze the effect of finite rate feedback on codedivision multipleaccess (CDMA) signature optimization and multipleinput multipleoutput (MIMO) beamforming vector selection. In CDMA signature optimization, for a particular user, the receiver selects a signature vector from a codebook ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
Abstract—We analyze the effect of finite rate feedback on codedivision multipleaccess (CDMA) signature optimization and multipleinput multipleoutput (MIMO) beamforming vector selection. In CDMA signature optimization, for a particular user, the receiver selects a signature vector from a codebook to best avoid interference from other users, and then feeds the corresponding index back to the specified user. For MIMO beamforming vector selection, the receiver chooses a beamforming vector from a given codebook to maximize the instantaneous information rate, and feeds back the corresponding index to the transmitter. These two problems are dual: both can be modeled as selecting a unit norm vector from a finite size codebook to “match ” a randomly generated Gaussian matrix. Assuming that the feedback link is rate limited, our main result is an exact asymptotic performance formula where the length of the signature/beamforming vector, the dimensions of interference/channel matrix, and the feedback rate approach infinity with constant ratios. The proof rests on the large deviations of the underlying random matrix ensemble. Further, we show that random codebooks generated from the isotropic distribution are asymptotically optimal not only on average, but also in probability. Index Terms—Beamforming, codedivision multiple access (CDMA), finite rate feedback, large deviations, multipleinput multipleoutput (MIMO), random matrix theory, signature optimization. I.
On robust regression with highdimensional predictors
, 2012
"... We consider the problem of understanding the properties of β = argmin β n∑ i=1 ρ(yi − X ′ iβ) , yi = X ′ iβ0 + ɛi, where Xi is a pdimensional vector of observed predictors, yi is a 1dimensional response and ρ is a given and known convex (loss) function. We are concerned with the highdimensional c ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
We consider the problem of understanding the properties of β = argmin β n∑ i=1 ρ(yi − X ′ iβ) , yi = X ′ iβ0 + ɛi, where Xi is a pdimensional vector of observed predictors, yi is a 1dimensional response and ρ is a given and known convex (loss) function. We are concerned with the highdimensional case where p/n has a finite nonzero limit. This problem is central to understanding the behavior of robust regression estimators in highdimension, something that appears to not have been studied before in statistics. Our analysis in this paper is heuristic but grounded in rigorous methods and relies principally on the concentration of measure phenomenon. Our derivations reveal the importance of the geometry of Xi’s in the behavior of β a key to understanding the robustness of our results. In the case where Xi are i.i.d N (0, Idp), β0 = 0, and ɛi are i.i.d, our work leads to the following conjecture/heuristic result: ‖ β ‖ is asymptotically deterministic and if ˆzɛ = ɛ+ ‖ β‖N (0, 1), where ɛ has the same distribution as ɛi, ‖ β ‖ has the property that, asymptotically as p and n grow to infinity (while p ≤ n and lim sup p/n < 1), E [proxc(ρ)] ′ (ˆzɛ) ) = 1 − p n, ‖ β‖2) = E ( [ˆzɛ − proxc(ρ)(ˆzɛ)] 2),
On the realized risk of highdimensional Markowitz portfolios
"... We study the realized risk of Markowitz portfolio computed using parameters estimated from data and generalizations to similar questions involving the outofsample risk in quadratic programs with linear equality constraints. We do so under the assumption that the data is generated according to an e ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
We study the realized risk of Markowitz portfolio computed using parameters estimated from data and generalizations to similar questions involving the outofsample risk in quadratic programs with linear equality constraints. We do so under the assumption that the data is generated according to an elliptical model, which allows us to study models where we have heavytails, tail dependence, and leptokurtic marginals for the data. We place ourselves in the setting of highdimensional inference where the number of assets in the portfolio, p, is large and comparable to the number of samples, n, we use to estimate the parameters. Our approach is based on random matrix theory. We consider both the impact of the estimation of the mean and of the covariance. Our work shows that risk is underestimated in this setting, and further, that in the class of elliptical distributions, the Gaussian case yields the least amount of risk underestimation. The problem is more pronounced for genuinely elliptical distributions and Gaussian computations give an overoptimistic view of the situation. We also propose a robust estimator of realized risk and investigate its performance in simulations. 1
On information plus noise kernel random matrices
, 2009
"... Kernel random matrices have attracted a lot of interest in recent years, from both practical and theoretical standpoints. Most of the theoretical work so far has focused on the case were the data is sampled from a lowdimensional structure. Very recently, the first results concerning kernel random m ..."
Abstract
 Add to MetaCart
Kernel random matrices have attracted a lot of interest in recent years, from both practical and theoretical standpoints. Most of the theoretical work so far has focused on the case were the data is sampled from a lowdimensional structure. Very recently, the first results concerning kernel random matrices with highdimensional input data were obtained, in a setting where the data was sampled from a genuinely highdimensional structure. In this paper, we consider the case where the data is of the type “information+noise”. In other words, each observation is the sum of two independent elements: one sampled from a “lowdimensional ” structure, the signal part of the data, the other being highdimensional noise, normalized to not overwhelm but still affect the signal. We show that in this setting the spectral properties of kernel random matrices can be understood from a new kernel matrix, computed only from the signal part of the data, but using (in general) a slightly different kernel. The Gaussian kernel has some special properties in this setting. 1
1 Robust MEstimation for Array Processing: A Random Matrix Approach
"... Abstract—This article studies the limiting behavior of a robust Mestimator of population covariance matrices as both the number of available samples and the population size are large. Using tools from random matrix theory, we prove that the difference between the sample covariance matrix and (a sca ..."
Abstract
 Add to MetaCart
Abstract—This article studies the limiting behavior of a robust Mestimator of population covariance matrices as both the number of available samples and the population size are large. Using tools from random matrix theory, we prove that the difference between the sample covariance matrix and (a scaled version of) the robust Mestimator tends to zero in spectral norm, almost surely. This result is applied to prove that recent subspace methods arising from random matrix theory can be made robust without altering their first order behavior. I.
1 The Effect of Finite Rate Feedback on CDMA Signature Optimization and MIMO Beamforming Vector Selection ∗
"... We analyze the effect of finite rate feedback on CDMA (codedivision multiple access) signature optimization and MIMO (multiinputmultioutput) beamforming vector selection. In CDMA signature optimization, for a particular user, the receiver selects a signature vector from a codebook to best avoid ..."
Abstract
 Add to MetaCart
We analyze the effect of finite rate feedback on CDMA (codedivision multiple access) signature optimization and MIMO (multiinputmultioutput) beamforming vector selection. In CDMA signature optimization, for a particular user, the receiver selects a signature vector from a codebook to best avoid interference from other users, and then feeds the corresponding index back to the specified user. For MIMO beamforming vector selection, the receiver chooses a beamforming vector from a given codebook to maximize the instantaneous information rate, and feeds back the corresponding index to the transmitter. These two problems are dual: both can be modeled as selecting a unit norm vector from a finite size codebook to “match ” a randomly generated Gaussian matrix. Assuming that the feedback link is rate limited, our main result is an exact asymptotic performance formula where the length of the signature/beamforming vector, the dimensions of interference/channel matrix, and the feedback rate approach infinity with constant ratios. The proof rests on the large deviations of the underlying random matrix ensemble. Further, we show that random codebooks generated from the isotropic distribution are asymptotically optimal not only on average, but also in probability. I.