## Robust Lasso with missing and grossly corrupted

Citations: | 7 - 0 self |

### BibTeX

@MISC{Nguyen_robustlasso,

author = {Nam H. Nguyen and Trac D. Tran Senior Member},

title = {Robust Lasso with missing and grossly corrupted},

year = {}

}

### OpenURL

### Abstract

observations

### Citations

1863 | Regression shrinkage and selection via the lasso
- Tibshirani
- 1996
(Show Context)
Citation Context ...s not consistent. Accordingly, there have been various lines of work on high dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. =-=[1]-=-, [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard met... |

1318 | Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information
- Candès, Romberg, et al.
- 2006
(Show Context)
Citation Context ... small portion of n. These conditions is of course far from the optimal bound in compressed sensing (CS) and statistics literature (recall k ≤ O(n/ log p) is sufficient in conventional analysis (e.g. =-=[30]-=-, [8]). Another line of work has also focused on the optimization (7). In both Laska et al. [19] and Li et al., [21], the authors establish that for Gaussian design matrix X, if n ≥ C(k + s) log p whe... |

751 |
Stable Signal recovery from incomplete and inaccurate measurements
- Candès, Romberg, et al.
(Show Context)
Citation Context ...and c2, we get 1 ∥ max √ ∥XSiTj ∥ ≤ ij n √ (√ k Cmax n + √ ) s ′ + τ . n with probability at least 1 − exp(−(τ 2 /2 − c1 − c2)n) where we recall the definition of Cmax := λmax(Σ). A standard bound in =-=[36]-=- gives us: ∑q i=3 ‖hTi‖2 ≤ k−1/2 ‖hT c‖1 . In addition, since h belongs to the set C, ‖hT c‖1 ≤ 3√k ‖h‖2 + 3λ √ s ‖f‖2 . Hence, q∑ i=1 ‖hTi ‖ 2 ≤ 2 ‖h‖ 2 + q∑ i=3 ij ‖hTi ‖ 2 ≤ 5 ‖h‖ 2 + 3λ √ s k ‖f‖ ... |

510 | Model selection and estimation in regression with grouped variables
- Yuan, Lin
- 2006
(Show Context)
Citation Context ...istent. Accordingly, there have been various lines of work on high dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. [1], [2], =-=[3]-=-, [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard method, namel... |

426 | The Dantzig selector: Statistical estimation when p is much larger than n. Annals of Statistics 35:2313–23516
- Candès, Tao
- 2007
(Show Context)
Citation Context ... consistent. Accordingly, there have been various lines of work on high dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. [1], =-=[2]-=-, [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard method, ... |

384 | High-dimensional graphs and variable selection with the Lasso
- Meinshausen, Bühlmann
(Show Context)
Citation Context ... there have been various lines of work on high dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. [1], [2], [3], [4], [5], [6], =-=[7]-=-, [8], [9], [10], [11], [12], [13], [14], [15]). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard method, namely Lasso [1], was pro... |

328 | Robust face recognition via sparse representation
- Wright, Yang, et al.
- 2009
(Show Context)
Citation Context ...y recovering data under gross error has gained increasing attentions recently with many interestingIEEE TRANSACTIONS ON INFORMATION THEORY, VOL. XXX, NO. XXX, XXX 2011 2 practical applications (e.g. =-=[17]-=-, [18], [19]) as well as theoretical consideration (e.g. [20], [21], [22], [23]). Another recent line of research on recovering the data from grossly corrupted measurements has been also studied in th... |

326 |
The Concentration of Measure Phenomenon
- Ledoux
- 2001
(Show Context)
Citation Context ... at least 1 − exp(−k/2) X ∥ ∗ √ X − Σ k n ∥ ≤ 4σmax(Σ) (66) n ( ∗ X X ∥ n ) −1 − Σ −1 ∥ ≤ √ 4 k . (67) σmin(Σ) n Finally, the following lemma states an useful concentration inequality on Haar measure =-=[44]-=-. Lemma 15. Support k < n and let f : Rn×k ↦→ R with Lipschitz norm ‖f‖ L = sup X̸=Y f(X) − f(Y ) . X − Y Then if U is distributed according to the Haar measure, P(f(U) ≥ median(f) + τ) ≤ exp(− REFERE... |

302 | Just relax: Convex programming methods for identifying sparse signals
- Tropp
(Show Context)
Citation Context ...ers small prediction error while others [10], [6] [11] seek to produce a regressor with minimal parameter estimation error, which is measured by the ℓ2-norm of ( ̂ β−β ⋆ ). Another line of work (e.g. =-=[16]-=-, [5], [8]) considers the variable selection in which the goal is to obtain an estimate that correctly identifies the support of the true regression vector. To achieve low prediction or parameter esti... |

222 | On model selection consistency of Lasso
- Zhao, Yu
(Show Context)
Citation Context ...cordingly, there have been various lines of work on high dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. [1], [2], [3], [4], =-=[5]-=-, [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard method, namely Lasso [1... |

184 | Simultaneous analysis of Lasso and Dantzig selector
- Bickel, Ritov, et al.
(Show Context)
Citation Context ...n various lines of work on high dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. [1], [2], [3], [4], [5], [6], [7], [8], [9], =-=[10]-=-, [11], [12], [13], [14], [15]). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard method, namely Lasso [1], was proposed to use l1-... |

162 | Sharp thresholds for high-dimensional and noisy sparsity recovery using ℓ1-constrained quadratic programming (Lasso
- Wainwright
- 2009
(Show Context)
Citation Context ...e have been various lines of work on high dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. [1], [2], [3], [4], [5], [6], [7], =-=[8]-=-, [9], [10], [11], [12], [13], [14], [15]). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard method, namely Lasso [1], was proposed... |

142 | Robust principal component analysis
- Candès, Li, et al.
(Show Context)
Citation Context ... [21], [22], [23]). Another recent line of research on recovering the data from grossly corrupted measurements has been also studied in the context of robust principal component analysis (RPCA) (e.g. =-=[24]-=-, [25], [26]). Let us consider several examples as illustrations. • Face recognition. The model (3) has been proposed by Wright et al. [17] in the context of face recognition. In this problem, a face ... |

117 | Lasso-type recovery of sparse representations for high-dimensional data - Meinhausen, Yu |

91 |
Adaptive estimation of a quadratic functional by model selection
- Laurent, Massart
(Show Context)
Citation Context ... from a standard Gaussian bound. For a Gaussian variable Z ∼ N (0, σ2 ), we have with all τ > 0 ( 2 τ P(|Z| ≥ τ) ≤ 2 exp − 2σ2 ) . (61) The following tail bounds on the Chi-square variates taken from =-=[43]-=- are useful Lemma 13. Let X be a centralized χ2-variate with d degree of freedom. Then for all τ ∈ (0, 1/2), we have ( P (X ≥ d(1 + τ)) ≤ exp − 3 ) 2 dτ 16 ( P (X ≤ d(1 − τ)) ≤ exp − 1 ) 2 dτ . 4 We a... |

83 | Sparsity oracle inequalities for the Lasso
- Bunea, Tsybakov, et al.
- 2007
(Show Context)
Citation Context ...nes of work on high dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], =-=[12]-=-, [13], [14], [15]). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard method, namely Lasso [1], was proposed to use l1-penalty as a... |

77 | SpAM: sparse additive models
- Ravikumar, Liu, et al.
(Show Context)
Citation Context ...ore general setting is an interesting open problem. Lastly, although our current work focused exclusively on linear regression, it would be interesting to investigate the sparse additive models (e.g. =-=[41]-=-, [42]) under grossly corrupted observations. A. Proof of Lemma 5 VIII. APPENDIX Decomposing XScT as XScT = WScT ΣT T where WScT ∈ R (n−s)×k is the random matrix with i.i.d. normal Gaussian entries, w... |

74 | Sparse subspace clustering
- Elhamifar, Vidal
- 2009
(Show Context)
Citation Context ...vering data under gross error has gained increasing attentions recently with many interestingIEEE TRANSACTIONS ON INFORMATION THEORY, VOL. XXX, NO. XXX, XXX 2011 2 practical applications (e.g. [17], =-=[18]-=-, [19]) as well as theoretical consideration (e.g. [20], [21], [22], [23]). Another recent line of research on recovering the data from grossly corrupted measurements has been also studied in the cont... |

72 | A unified framework for high-dimensional analysis of M-estimatorswith decomposable regularizers
- Negahban, Ravikumar, et al.
(Show Context)
Citation Context ... R n | ‖hT c‖ 1 + λ ‖fS c‖ 1 ≤ 3 ‖hT ‖ 1 + 3λ ‖fS‖ 1 }. (11) This assumption is a natural extension of the restricted eigenvalue condition and restricted strong convexity considered in [10] ,[33] and =-=[34]-=-. In the absence of a vector f in the equation (10) and in the set C, this condition returns to the restricted eigenvalue defined in [10]. As discussed in more detail in [10] and [35], restricted eige... |

69 | The composite absolute penalties family for grouped and hierarchical variable selection. Ann.Stat.,37(6A):3468–3497
- Zhao, Rocha, et al.
- 2009
(Show Context)
Citation Context ...t. Accordingly, there have been various lines of work on high dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. [1], [2], [3], =-=[4]-=-, [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard method, namely Las... |

62 | The benefit of group sparsity
- Huang, Zhang
(Show Context)
Citation Context ...h dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], =-=[15]-=-). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard method, namely Lasso [1], was proposed to use l1-penalty as a surrogate functio... |

49 |
de Geer. Taking advantage of sparsity in multi-task learning
- Lounici, Pontil, et al.
- 2009
(Show Context)
Citation Context ...n be extended to robust group/multivariate Lasso model. This model has been shown to outperform the conventional Lasso in many practical applications as well as theoretical analysis (e.g. [14], [15], =-=[38]-=-, [39]). It would be interesting to obtain the upper and lower bound of the sample size when a significant fraction of observations is corrupted in this setting. Another interesting direction is to co... |

41 | Some sharp performance bounds for least squares regression with penalization - Zhang - 2009 |

33 | Robust PCA via outlier pursuit
- Hu, Caramanis, et al.
- 2010
(Show Context)
Citation Context ... [22], [23]). Another recent line of research on recovering the data from grossly corrupted measurements has been also studied in the context of robust principal component analysis (RPCA) (e.g. [24], =-=[25]-=-, [26]). Let us consider several examples as illustrations. • Face recognition. The model (3) has been proposed by Wright et al. [17] in the context of face recognition. In this problem, a face test s... |

31 | High-dimensional additive modeling
- Meier, Geer, et al.
(Show Context)
Citation Context ...neral setting is an interesting open problem. Lastly, although our current work focused exclusively on linear regression, it would be interesting to investigate the sparse additive models (e.g. [41], =-=[42]-=-) under grossly corrupted observations. A. Proof of Lemma 5 VIII. APPENDIX Decomposing XScT as XScT = WScT ΣT T where WScT ∈ R (n−s)×k is the random matrix with i.i.d. normal Gaussian entries, we have... |

30 | On the conditions used to prove oracle results for the lasso
- Geer, Buhlmann
(Show Context)
Citation Context ...n [10] ,[33] and [34]. In the absence of a vector f in the equation (10) and in the set C, this condition returns to the restricted eigenvalue defined in [10]. As discussed in more detail in [10] and =-=[35]-=-, restricted eigenvalue is among the weakest assumption on the design matrix such that the solution of the Lasso is consistent. With this assumption at hand, we now state the first theorem Theorem 1. ... |

29 |
Compressed sensing for networked data
- Haupt, Rabbat, et al.
- 2008
(Show Context)
Citation Context ...rably similar. Sensor network. In this model, a network of sensors collect measurements of a signal β⋆ independently by simply projecting β⋆ onto the row vectors of a sensing matrix X, yi = 〈Xi, β⋆ 〉 =-=[27]-=-. The measurements yi are then sent to the central hub for analysis. However, it is highly likely that a small percentage of sensors might fail to send the measurements correctly and sometimes even re... |

28 | Support union recovery in high-dimensional multivariate regression
- Obozinski, Wainwright, et al.
(Show Context)
Citation Context ...on high dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], =-=[14]-=-, [15]). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard method, namely Lasso [1], was proposed to use l1-penalty as a surrogate f... |

19 | Near-ideal model selection by l1 minimization - Candès, Plan |

19 | High-dimensional Regression With Noisy And Missing Data: Provable Guarantees With Nonconvexity
- Loh, Wainwright
(Show Context)
Citation Context ...ted in this setting. Another interesting direction is to consider a more general situation where both the observations and the data matrix are corrupted/missing. In a recent paper, Loh and Wainwright =-=[40]-=- established the consistency of the Lasso with noisy/corrupted/missing data matrix. Whether similar results would hold for more general setting is an interesting open problem. Lastly, although our cur... |

17 | Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions
- Agarwal, Negahban, et al.
(Show Context)
Citation Context ... [23]). Another recent line of research on recovering the data from grossly corrupted measurements has been also studied in the context of robust principal component analysis (RPCA) (e.g. [24], [25], =-=[26]-=-). Let us consider several examples as illustrations. • Face recognition. The model (3) has been proposed by Wright et al. [17] in the context of face recognition. In this problem, a face test sample ... |

17 |
Robust regression shrinkage and consistent variable selection through the LAD-Lasso
- Wang, Li, et al.
- 2007
(Show Context)
Citation Context ...obust to noise. However, no theoretical analysis is provided in the paper. In another direction, the problem of robust Lasso under corrupted observations is also carefully investigated by Wang et al. =-=[29]-=-. In this appealing paper, instead of using the quadratic loss function as in Lasso, the authors propose to employ LAD-Lasso criterion: min β ‖y − Xβ‖ 1 + p∑ λj|xj|. (8) j=1 This optimization combines... |

16 | Honest variable selection in linear and logistic regression models via ℓ1 and ℓ1 +ℓ2 penalization
- Bunea
(Show Context)
Citation Context ... work on high dimensional inference based on imposing different types of structure constraints such as sparsity and group sparsity (e.g. [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], =-=[13]-=-, [14], [15]). Among them, the most popular model focused on sparsity assumption of the regression vector. To estimate β, a standard method, namely Lasso [1], was proposed to use l1-penalty as a surro... |

16 |
Dense error correction via l1-minimization
- Wright, Ma
- 2010
(Show Context)
Citation Context ...entions recently with many interestingIEEE TRANSACTIONS ON INFORMATION THEORY, VOL. XXX, NO. XXX, XXX 2011 2 practical applications (e.g. [17], [18], [19]) as well as theoretical consideration (e.g. =-=[20]-=-, [21], [22], [23]). Another recent line of research on recovering the data from grossly corrupted measurements has been also studied in the context of robust principal component analysis (RPCA) (e.g.... |

14 | Simultaneous support recovery in high-dimensional regression: Benefits and perils of ℓ1,∞-regularization
- Negahban, Wainwright
- 2011
(Show Context)
Citation Context ...xtended to robust group/multivariate Lasso model. This model has been shown to outperform the conventional Lasso in many practical applications as well as theoretical analysis (e.g. [14], [15], [38], =-=[39]-=-). It would be interesting to obtain the upper and lower bound of the sample size when a significant fraction of observations is corrupted in this setting. Another interesting direction is to consider... |

12 | Exact signal recovery from sparsely corrupted measurements through the pursuit of justice
- Laska, Davenport, et al.
- 2009
(Show Context)
Citation Context ... data under gross error has gained increasing attentions recently with many interestingIEEE TRANSACTIONS ON INFORMATION THEORY, VOL. XXX, NO. XXX, XXX 2011 2 practical applications (e.g. [17], [18], =-=[19]-=-) as well as theoretical consideration (e.g. [20], [21], [22], [23]). Another recent line of research on recovering the data from grossly corrupted measurements has been also studied in the context of... |

9 | Restricted eigenvalue properties for correlated Gaussian designs
- RASKUTTI, WAINWRIGHT, et al.
- 2010
(Show Context)
Citation Context ...) ∈ R p × R n | ‖hT c‖ 1 + λ ‖fS c‖ 1 ≤ 3 ‖hT ‖ 1 + 3λ ‖fS‖ 1 }. (11) This assumption is a natural extension of the restricted eigenvalue condition and restricted strong convexity considered in [10] ,=-=[33]-=- and [34]. In the absence of a vector f in the equation (10) and in the set C, this condition returns to the restricted eigenvalue defined in [10]. As discussed in more detail in [10] and [35], restri... |

5 |
Compressed sensing and matrix completion with constant proportion of corruptions. Arxiv preprint arXiv:1104.1041
- Li
- 2011
(Show Context)
Citation Context ...ith many interestingIEEE TRANSACTIONS ON INFORMATION THEORY, VOL. XXX, NO. XXX, XXX 2011 2 practical applications (e.g. [17], [18], [19]) as well as theoretical consideration (e.g. [20], [21], [22], =-=[23]-=-). Another recent line of research on recovering the data from grossly corrupted measurements has been also studied in the context of robust principal component analysis (RPCA) (e.g. [24], [25], [26])... |

4 | On the systematic measurement matrix for compressed sensing in the presence of gross errors
- Li, Wu, et al.
- 2010
(Show Context)
Citation Context ...s recently with many interestingIEEE TRANSACTIONS ON INFORMATION THEORY, VOL. XXX, NO. XXX, XXX 2011 2 practical applications (e.g. [17], [18], [19]) as well as theoretical consideration (e.g. [20], =-=[21]-=-, [22], [23]). Another recent line of research on recovering the data from grossly corrupted measurements has been also studied in the context of robust principal component analysis (RPCA) (e.g. [24],... |

4 | Stable restoration and separation of approximately sparse signals
- Studer, Baraniuk
(Show Context)
Citation Context ...recovery of sparse vectors via ℓ1-minimization. These results, however, do not allow the fraction of corruption to come close to unity. Also related to our paper is recent work by Studer et al., [31] =-=[32]-=- in which the authors establish different results for deterministic design matrix. Among the previous work, the most closely related to our current paper are recent results by Li [23] and Nguyen et al... |

3 |
Exact recoverability from dense corrupted observations via l 1 minimization
- Nguyen, Tran
(Show Context)
Citation Context ...ntly with many interestingIEEE TRANSACTIONS ON INFORMATION THEORY, VOL. XXX, NO. XXX, XXX 2011 2 practical applications (e.g. [17], [18], [19]) as well as theoretical consideration (e.g. [20], [21], =-=[22]-=-, [23]). Another recent line of research on recovering the data from grossly corrupted measurements has been also studied in the context of robust principal component analysis (RPCA) (e.g. [24], [25],... |

3 | Regularization of case-specific parameters for robustness and efficiency
- Lee, MacEachern, et al.
- 2011
(Show Context)
Citation Context ...g the following linear program min β,e ‖β‖ 1 + ‖e‖ 1 s.t. y = Xβ + √ ne. (7)IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. XXX, NO. XXX, XXX 2011 3 From a different viewpoint, in the intriguing paper =-=[28]-=-, Lee et al. study a general loss function model. To obtain more flexibility in controlling the undesirable influence of the model, they introduce a case-specific parameter vector e ∈ R n for the obse... |

2 |
Sparse signal recovery from sparsely corrupted measurements
- Studer, Kuppinger, et al.
- 2011
(Show Context)
Citation Context ...xact recovery of sparse vectors via ℓ1-minimization. These results, however, do not allow the fraction of corruption to come close to unity. Also related to our paper is recent work by Studer et al., =-=[31]-=- [32] in which the authors establish different results for deterministic design matrix. Among the previous work, the most closely related to our current paper are recent results by Li [23] and Nguyen ... |