## Simultaneous analysis of Lasso and Dantzig selector (2009)

### Cached

### Download Links

Venue: | ANNALS OF STATISTICS |

Citations: | 184 - 6 self |

### BibTeX

@ARTICLE{Bickel09simultaneousanalysis,

author = {Peter J. Bickel and Ya'acov Ritov and Alexandre B. Tsybakov},

title = {Simultaneous analysis of Lasso and Dantzig selector},

journal = {ANNALS OF STATISTICS},

year = {2009},

volume = {37},

number = {4}

}

### Years of Citing Articles

### OpenURL

### Abstract

We show that, under a sparsity scenario, the Lasso estimator and the Dantzig selector exhibit similar behavior. For both methods, we derive, in parallel, oracle inequalities for the prediction risk in the general nonparametric regression model, as well as bounds on the ℓp estimation loss for 1 ≤ p ≤ 2 in the linear model when the number of variables can be much larger than the sample size.

### Citations

752 | Least angle regression
- Efron, Hastie, et al.
(Show Context)
Citation Context ...the corresponding Lasso estimator M∑ (2.2) ̂fL(x) = f̂βL (x) = ̂βj,Lfj (z). The criterion in (2.1) isconvexinβ, so that standard convex optimization procedures can be used to compute ̂βL. We refer to =-=[9, 10, 20, 21, 24]-=- and[16] for detailed discussion of these optimization problems and fast algorithms. A necessary and sufficient condition of the minimizer in (2.1) is that 0 belongs to the subdifferential of the conv... |

424 | The Dantzig selector: statistical estimation when p is much larger than n
- Candès, Tao
(Show Context)
Citation Context ...estimator of parameters in highdimensional linear regression when the number of variables can be much larger than the sample size [8, 9, 11, 17, 18, 20–22, 26] and[27]. Quite recently, Candes and Tao =-=[7]-=- have proposed a new estimate for such linear models, the Dantzig selector, for which they establish optimal ℓ2 rate properties under a sparsity scenario; that is, when the number of nonzero component... |

382 | High-dimensional graphs and variable selection with the lasso - Meinshausen, Bühlmann |

323 | Regression shrinkage and selection via the - Tibshirani - 1996 |

297 | Stable recovery of sparse overcomplete representations in the presence of noise
- Donoho, Elad, et al.
(Show Context)
Citation Context ...c| 0 and, if (4.1) holds, |δ T J0XT ∣ XδJ c|/n ≤|δ 0 J c|1 max ∣ /n 0 j∈J c 0 ≤ θs,1|δ J c 0 |1|δJ0 |2 √ ≤ c0θs,1 s|δJ0 |22 . ∣ δ T J0 XT x(j) Another type of assumption related to “mutual coherence” =-=[8]-=- is discussed in connection to Lasso in [4, 5]. We state it in two different forms, which are given below.1714 P. J. BICKEL, Y. RITOV AND A. B. TSYBAKOV ASSUMPTION 4. where c0 > 0 is a constant. (4.2... |

167 | Pathwise coordinate optimization
- Friedman, Hastie, et al.
- 2007
(Show Context)
Citation Context ...the corresponding Lasso estimator M∑ (2.2) ̂fL(x) = f̂βL (x) = ̂βj,Lfj (z). The criterion in (2.1) isconvexinβ, so that standard convex optimization procedures can be used to compute ̂βL. We refer to =-=[9, 10, 20, 21, 24]-=- and[16] for detailed discussion of these optimization problems and fast algorithms. A necessary and sufficient condition of the minimizer in (2.1) is that 0 belongs to the subdifferential of the conv... |

162 | A new approach to variable selection in least squares problems
- Osborne, Presnell, et al.
- 2000
(Show Context)
Citation Context ...the corresponding Lasso estimator M∑ (2.2) ̂fL(x) = f̂βL (x) = ̂βj,Lfj (z). The criterion in (2.1) isconvexinβ, so that standard convex optimization procedures can be used to compute ̂βL. We refer to =-=[9, 10, 20, 21, 24]-=- and[16] for detailed discussion of these optimization problems and fast algorithms. A necessary and sufficient condition of the minimizer in (2.1) is that 0 belongs to the subdifferential of the conv... |

144 | On the LASSO and Its Dual
- Osborne, Presnell, et al.
- 2000
(Show Context)
Citation Context |

140 | The group lasso for logistic regression
- Meier, Geer, et al.
(Show Context)
Citation Context ... estimator M∑ (2.2) ̂fL(x) = f̂βL (x) = ̂βj,Lfj (z). The criterion in (2.1) isconvexinβ, so that standard convex optimization procedures can be used to compute ̂βL. We refer to [9, 10, 20, 21, 24] and=-=[16]-=- for detailed discussion of these optimization problems and fast algorithms. A necessary and sufficient condition of the minimizer in (2.1) is that 0 belongs to the subdifferential of the convex funct... |

137 | Asymptotics for Lasso-type estimators - Knight, Fu - 2000 |

83 | Sparsity oracle inequalities for the Lasso - Bunea, Tsybakov, et al. |

79 |
Aggregation for Gaussian regression
- Bunea, Tsybakov, et al.
(Show Context)
Citation Context ... study these two estimators in parallel. For the fixed design Gaussian regression model, we recover, as particular cases, sparsity oracle inequalities for the Lasso, as in Bunea, Tsybakov and Wegkamp =-=[4]-=-, and ℓ2 bounds for the coefficients of Dantzig selector, as in Candes and Tao [7]. This is obtained as a consequence of our more general results, which are the following: • In the nonparametric regre... |

74 |
On model selection consistency of
- Zhao, Yu
- 2006
(Show Context)
Citation Context ...heℓ1 penalized least squares (Lasso) estimator of parameters in highdimensional linear regression when the number of variables can be much larger than the sample size [8, 9, 11, 17, 18, 20–22, 26] and=-=[27]-=-. Quite recently, Candes and Tao [7] have proposed a new estimate for such linear models, the Dantzig selector, for which they establish optimal ℓ2 rate properties under a sparsity scenario; that is, ... |

73 | Persistence in high-dimensional linear predictor selection and the virtue of overparametrization - Greenshtein, Ritov |

32 | A.: Functional aggregation for nonparametric estimation - Juditsky, Nemirovski |

23 |
Sparsity in penalized empirical risk minimization
- Koltchinskii
(Show Context)
Citation Context ...dified versions of Lasso estimators (nonquadratic terms and/or penalties slightly different from ℓ1) for nonparametric regression with random design are suggested and studied under prediction loss in =-=[14]-=- and[25]. Sparsity oracle inequalities for the Dantzig selector with random design are obtained in [15]. In linear fixed design regression, Meinshausen and Yu [18] establish a bound on the ℓ2 loss for... |

22 |
Dantzig selector and sparsity oracle inequalities
- Koltchinskii
- 2008
(Show Context)
Citation Context ...r nonparametric regression with random design are suggested and studied under prediction loss in [14] and[25]. Sparsity oracle inequalities for the Dantzig selector with random design are obtained in =-=[15]-=-. In linear fixed design regression, Meinshausen and Yu [18] establish a bound on the ℓ2 loss for the coefficients of Lasso that is quite different from the bound on the same loss for the Dantzig sele... |

21 |
Aggregation and Sparsity Via ℓ1 Penalized Least Squares. Learning theory
- Bunea, Tsybakov, et al.
- 2006
(Show Context)
Citation Context ...for all 1 <p≤ 2, we have | ̂βL − β ∗ | p { √ s (7.10) p ≤ 16 1 + 3 m } 2(p−1) s ( Aσ κ 2 (s, m, 3) √ ) p log M . n Inequalities of the form similar to (7.7)and(7.8) can be deduced from the results of =-=[3]-=- under more restrictive conditions on the Gram matrix (the mutual coherence assumption, cf. Assumption 5 of Section 4). Assumptions RE(s, 1) and RE(s, 3), respectively, can be dropped in Theorems 7.1 ... |

19 |
High-dimensional generalized linear models and the
- GEER
- 2008
(Show Context)
Citation Context ...ersions of Lasso estimators (nonquadratic terms and/or penalties slightly different from ℓ1) for nonparametric regression with random design are suggested and studied under prediction loss in [14] and=-=[25]-=-. Sparsity oracle inequalities for the Dantzig selector with random design are obtained in [15]. In linear fixed design regression, Meinshausen and Yu [18] establish a bound on the ℓ2 loss for the coe... |

13 | Topics in Non-parametric Statistics. In: Ecole d’Eté de Probabilités de Saint-Flour - Nemirovski - 2000 |

13 |
On algorithms for solving least squares problems under an L1 penalty or an L1 constraint
- Turlach
- 2005
(Show Context)
Citation Context |

12 | Aggregation for regression learning - Bunea, Tsybakov, et al. - 2004 |

11 | Model-selection consistency of the Lasso in high-dimensional linear regression - Zhang, Huang - 2006 |

9 | Sparse density estimation with ℓ1 penalties
- Bunea, Tsybakov, et al.
(Show Context)
Citation Context ...tion in classical nonparametric regression settings, as well as for the problem of aggregation of estimators. An analog of Lasso for density estimation with similar properties (SPADES) is proposed in =-=[6]-=-. Modified versions of Lasso estimators (nonquadratic terms and/or penalties slightly different from ℓ1) for nonparametric regression with random design are suggested and studied under prediction loss... |

7 |
Lasso type recovery of sparse representations for high dimensional data
- Meinshausen, Yu
- 2006
(Show Context)
Citation Context ...and studied under prediction loss in [14] and[25]. Sparsity oracle inequalities for the Dantzig selector with random design are obtained in [15]. In linear fixed design regression, Meinshausen and Yu =-=[18]-=- establish a bound on the ℓ2 loss for the coefficients of Lasso that is quite different from the bound on the same loss for the Dantzig selector proven in [7]. The main message of this paper is that, ... |

5 | Discussion of The Dantzig selector: statistical estimation when p is much larger than n, by E.J
- Bickel
- 2007
(Show Context)
Citation Context ... in Appendix A. There exist other sufficient conditions for Assumptions RE(s, c0) and RE(s, m, c0) to hold. We mention here three of them implying Assumption RE(s, c0). The first one is the following =-=[1]-=-. ASSUMPTION 3. For an integers such that 1 ≤ s ≤ M, wehave √ φmin(s) > 2c0θs,1 s, where c0 > 0 is a constant. To argue that Assumption 3 implies RE(s, c0), it suffices to remark that 1 n |Xδ|2 1 2 ≥ ... |

2 |
Comments on “Regularization in statistics”, by
- Tsybakov
- 2006
(Show Context)
Citation Context ...ion losses of the Dantzig and Lasso estimators is of the same order as the distances between them and their oracle approximations. A general discussion of sparsity oracle inequalities can be found in =-=[23]-=-. Such inequalities have been recently obtained for the Lasso type estimators in a number of settings [2–6, 14]and[25]. In particular, the regression model with fixed design that we study here is cons... |