DMCA
Minimax rates of estimation for high-dimensional linear regression over -balls (2009)
Cached
Download Links
| Citations: | 96 - 19 self |
BibTeX
@MISC{Raskutti09minimaxrates,
author = {Garvesh Raskutti and Martin J. Wainwright and Senior Member},
title = {Minimax rates of estimation for high-dimensional linear regression over -balls},
year = {2009}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract—Consider the high-dimensional linear regression model,where is an observation vector, is a design matrix with, is an unknown regression vector, and is additive Gaussian noise. This paper studies the minimax rates of convergence for estimating in either-loss and-prediction loss, assuming that belongs to an-ball for some.Itisshown that under suitable regularity conditions on the design matrix, the minimax optimal rate in-loss and-prediction loss scales as. The analysis in this paper reveals that conditions on the design matrix enter into the rates for-error and-prediction error in complementary ways in the upper and lower bounds. Our proofs of the lower bounds are information theoretic in nature, based on Fano’s inequality and results on the metric entropy of the balls, whereas our proofs of the upper bounds are constructive, involving direct analysis of least squares over-balls. For the special case, corresponding to models with an exact sparsity constraint, our results show that although computationally efficient-based methods can achieve the minimax rates up to constant factors, they require slightly stronger assumptions on the design matrix than optimal algorithms involving least-squares over the-ball. Index Terms—Compressed sensing, minimax techniques, regression analysis. I.
Keyphrases
minimax rate design matrix high-dimensional linear regression observation vector regression analysis direct analysis special case additive gaussian noise exact sparsity constraint unknown regression vector constant factor paper study rate for-error and-prediction error index term minimax technique metric entropy fano inequality efficient-based method square over-balls abstract consider high-dimensional linear regression model design matrix enter either-loss and-prediction loss suitable regularity condition upper bound complementary way
