Results 11  20
of
450
Lassotype recovery of sparse representations for highdimensional data
 ANNALS OF STATISTICS
, 2009
"... The Lasso is an attractive technique for regularization and variable selection for highdimensional data, where the number of predictor variables pn is potentially much larger than the number of samples n. However, it was recently discovered that the sparsity pattern of the Lasso estimator can only ..."
Abstract

Cited by 245 (14 self)
 Add to MetaCart
(Show Context)
The Lasso is an attractive technique for regularization and variable selection for highdimensional data, where the number of predictor variables pn is potentially much larger than the number of samples n. However, it was recently discovered that the sparsity pattern of the Lasso estimator can only be asymptotically identical to the true sparsity pattern if the design matrix satisfies the socalled irrepresentable condition. The latter condition can easily be violated in the presence of highly correlated variables. Here we examine the behavior of the Lasso estimators if the irrepresentable condition is relaxed. Even though the Lasso cannot recover the correct sparsity pattern, we show that the estimator is still consistent in the ℓ2norm sense for fixed designs under conditions on (a) the number sn of nonzero components of the vector βn and (b) the minimal singular values of design matrices that are induced by selecting small subsets of variables. Furthermore, a rate of convergence result is obtained on the ℓ2 error with an appropriate choice of the smoothing parameter. The rate is shown to be
Compressive Sensing and Structured Random Matrices
 RADON SERIES COMP. APPL. MATH XX, 1–95 © DE GRUYTER 20YY
, 2011
"... These notes give a mathematical introduction to compressive sensing focusing on recovery using ℓ1minimization and structured random matrices. An emphasis is put on techniques for proving probabilistic estimates for condition numbers of structured random matrices. Estimates of this type are key to ..."
Abstract

Cited by 156 (18 self)
 Add to MetaCart
These notes give a mathematical introduction to compressive sensing focusing on recovery using ℓ1minimization and structured random matrices. An emphasis is put on techniques for proving probabilistic estimates for condition numbers of structured random matrices. Estimates of this type are key to providing conditions that ensure exact or approximate recovery of sparse vectors using ℓ1minimization.
Analysis versus synthesis in signal priors
, 2005
"... The concept of prior probability for signals plays a key role in the successful solution of many inverse problems. Much of the literature on this topic can be divided between analysisbased and synthesisbased priors. Analysisbased priors assign probability to a signal through various forward measu ..."
Abstract

Cited by 147 (16 self)
 Add to MetaCart
(Show Context)
The concept of prior probability for signals plays a key role in the successful solution of many inverse problems. Much of the literature on this topic can be divided between analysisbased and synthesisbased priors. Analysisbased priors assign probability to a signal through various forward measurements of it, while synthesisbased priors seek a reconstruction of the signal as a combination of atom signals. In this paper we describe these two prior classes, focusing on the distinction between them. We show that although when reducing to the complete and undercomplete formulations the two become equivalent, in their more interesting overcomplete formulation the two types depart. Focusing on the ℓ1 denoising case, we present several ways of comparing the two types of priors, establishing the existence of an unbridgeable gap between them. 1.
Aggregation for Gaussian regression
 SUBMITTED TO THE ANNALS OF STATISTICS
, 2007
"... This paper studies statistical aggregation procedures in the regression setting. A motivating factor is the existence of many different methods of estimation, leading to possibly competing estimators. We consider here three different types of aggregation: model selection (MS) aggregation, convex (C) ..."
Abstract

Cited by 144 (17 self)
 Add to MetaCart
(Show Context)
This paper studies statistical aggregation procedures in the regression setting. A motivating factor is the existence of many different methods of estimation, leading to possibly competing estimators. We consider here three different types of aggregation: model selection (MS) aggregation, convex (C) aggregation and linear (L) aggregation. The objective of (MS) is to select the optimal single estimator from the list; that of (C) is to select the optimal convex combination of the given estimators; and that of (L) is to select the optimal linear combination of the given estimators. We are interested in evaluating the rates of convergence of the excess risks of the estimators obtained by these procedures. Our approach is motivated by recent minimax results in [34, 40]. There exist competing aggregation procedures achieving optimal convergence rates for each of the (MS), (C) and (L) cases separately. Since these procedures are not directly comparable with each other, we suggest an alternative solution. We prove that all the three optimal rates, as well as those for the newly introduced (S)
Theoretical results on sparse representations of multiplemeasurement vectors
 IEEE Trans. Signal Process
, 2006
"... Abstract — Multiple measurement vector (MMV) is a relatively new problem in sparse representations. Efficient methods have been proposed. Considering many theoretical results that are available in a simple case – single measure vector (SMV) – the theoretical analysis regarding MMV is lacking. In th ..."
Abstract

Cited by 142 (2 self)
 Add to MetaCart
(Show Context)
Abstract — Multiple measurement vector (MMV) is a relatively new problem in sparse representations. Efficient methods have been proposed. Considering many theoretical results that are available in a simple case – single measure vector (SMV) – the theoretical analysis regarding MMV is lacking. In this paper, some known results of SMV are generalized to MMV. Some of these new results take advantages of additional information in the formulation of MMV. We consider the uniqueness under both an ℓ0norm like criterion and an ℓ1norm like criterion. The consequent equivalence between the ℓ0norm approach and the ℓ1norm approach indicates a computationally efficient way of finding the sparsest representation in an overcomplete dictionary. For greedy algorithms, it is proven that under certain conditions, orthogonal matching pursuit (OMP) can find the sparsest representation of an MMV with computational efficiency, just like in SMV. Simulations show that the predictions made by the proved theorems tend to be very conservative; this is consistent with some recent theoretical advances in probability. The connections will be discussed.
Compressed Sensing and Redundant Dictionaries
"... This article extends the concept of compressed sensing to signals that are not sparse in an orthonormal basis but rather in a redundant dictionary. It is shown that a matrix, which is a composition of a random matrix of certain type and a deterministic dictionary, has small restricted isometry con ..."
Abstract

Cited by 137 (13 self)
 Add to MetaCart
(Show Context)
This article extends the concept of compressed sensing to signals that are not sparse in an orthonormal basis but rather in a redundant dictionary. It is shown that a matrix, which is a composition of a random matrix of certain type and a deterministic dictionary, has small restricted isometry constants. Thus, signals that are sparse with respect to the dictionary can be recovered via Basis Pursuit from a small number of random measurements. Further, thresholding is investigated as recovery algorithm for compressed sensing and conditions are provided that guarantee reconstruction with high probability. The different schemes are compared by numerical experiments.
Recovery of exact sparse representations in the presence of bounded noise
 IEEE Trans. on I.T
, 2005
"... The purpose of this contribution is to extend some recent results on sparse representations of signals in redundant bases developed in the noisefree case to the case of noisy observations. The type of questions addressed so far is: given a (n,m)matrix with and a vector, find a sufficient condition ..."
Abstract

Cited by 133 (5 self)
 Add to MetaCart
The purpose of this contribution is to extend some recent results on sparse representations of signals in redundant bases developed in the noisefree case to the case of noisy observations. The type of questions addressed so far is: given a (n,m)matrix with and a vector, find a sufficient condition for to have an unique sparsest representation as a linear combination of the columns of. The answer is a bound on the number of nonzero entries of say, that guaranties that is the unique and sparsest solution of with. We consider the case where satisfies the sparsity conditions requested in the noisefree case and seek conditions on, a vector of additive noise or modeling errors, under which can be recovered from in a sense to be defined. 1.
Fields of Experts
 INTERNATIONAL JOURNAL OF COMPUTER VISION
, 2008
"... We develop a framework for learning generic, expressive image priors that capture the statistics of natural scenes and can be used for a variety of machine vision tasks. The approach provides a practical method for learning highorder Markov random field (MRF) models with potential functions that ex ..."
Abstract

Cited by 130 (12 self)
 Add to MetaCart
We develop a framework for learning generic, expressive image priors that capture the statistics of natural scenes and can be used for a variety of machine vision tasks. The approach provides a practical method for learning highorder Markov random field (MRF) models with potential functions that extend over large pixel neighborhoods. These clique potentials are modeled using the ProductofExperts framework that uses nonlinear functions of many linear filter responses. In contrast to previous MRF approaches all parameters, including the linear filters themselves, are learned from training data. We demonstrate the capabilities of this FieldofExperts model with two example applications, image denoising and image inpainting, which are implemented using a simple, approximate inference scheme. While the model is trained on a generic image database and is not tuned toward a specific application, we obtain results that compete with specialized techniques.
Greedy solution of illposed problems: Error bounds and exact inversion
, 2009
"... ..."
(Show Context)
Geometric approach to error correcting codes and reconstruction of signals
 INT. MATH. RES. NOT
, 2005
"... ..."
(Show Context)