Results 1  10
of
31
Compressed sensing
 IEEE Trans. Inform. Theory
"... Abstract—Suppose is an unknown vector in (a digital image or signal); we plan to measure general linear functionals of and then reconstruct. If is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measureme ..."
Abstract

Cited by 1730 (18 self)
 Add to MetaCart
Abstract—Suppose is an unknown vector in (a digital image or signal); we plan to measure general linear functionals of and then reconstruct. If is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measurements can be dramatically smaller than the size. Thus, certain natural classes of images with pixels need only = ( 1 4 log 5 2 ()) nonadaptive nonpixel samples for faithful recovery, as opposed to the usual pixel samples. More specifically, suppose has a sparse representation in some orthonormal basis (e.g., wavelet, Fourier) or tight frame (e.g., curvelet, Gabor)—so the coefficients belong to an ball for 0 1. The most important coefficients in that expansion allow reconstruction with 2 error ( 1 2 1
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
, 2007
"... A fullrank matrix A ∈ IR n×m with n < m generates an underdetermined system of linear equations Ax = b having infinitely many solutions. Suppose we seek the sparsest solution, i.e., the one with the fewest nonzero entries: can it ever be unique? If so, when? As optimization of sparsity is combinato ..."
Abstract

Cited by 202 (31 self)
 Add to MetaCart
A fullrank matrix A ∈ IR n×m with n < m generates an underdetermined system of linear equations Ax = b having infinitely many solutions. Suppose we seek the sparsest solution, i.e., the one with the fewest nonzero entries: can it ever be unique? If so, when? As optimization of sparsity is combinatorial in nature, are there efficient methods for finding the sparsest solution? These questions have been answered positively and constructively in recent years, exposing a wide variety of surprising phenomena; in particular, the existence of easilyverifiable conditions under which optimallysparse solutions can be found by concrete, effective computational methods. Such theoretical results inspire a bold perspective on some important practical problems in signal and image processing. Several wellknown signal and image processing problems can be cast as demanding solutions of undetermined systems of equations. Such problems have previously seemed, to many, intractable. There is considerable evidence that these problems often have sparse solutions. Hence, advances in finding sparse solutions to underdetermined systems energizes research on such signal and image processing problems – to striking effect. In this paper we review the theoretical results on sparse solutions of linear systems, empirical
For most large underdetermined systems of equations, the minimal l1norm nearsolution approximates the sparsest nearsolution
 Comm. Pure Appl. Math
, 2004
"... We consider inexact linear equations y ≈ Φα where y is a given vector in R n, Φ is a given n by m matrix, and we wish to find an α0,ɛ which is sparse and gives an approximate solution, obeying �y − Φα0,ɛ�2 ≤ ɛ. In general this requires combinatorial optimization and so is considered intractable. On ..."
Abstract

Cited by 68 (0 self)
 Add to MetaCart
We consider inexact linear equations y ≈ Φα where y is a given vector in R n, Φ is a given n by m matrix, and we wish to find an α0,ɛ which is sparse and gives an approximate solution, obeying �y − Φα0,ɛ�2 ≤ ɛ. In general this requires combinatorial optimization and so is considered intractable. On the other hand, the ℓ 1 minimization problem min �α�1 subject to �y − Φα�2 ≤ ɛ, is convex, and is considered tractable. We show that for most Φ the solution ˆα1,ɛ = ˆα1,ɛ(y, Φ) of this problem is quite generally a good approximation for ˆα0,ɛ. We suppose that the columns of Φ are normalized to unit ℓ 2 norm 1 and we place uniform measure on such Φ. We study the underdetermined case where m ∼ An, A> 1 and prove the existence of ρ = ρ(A) and C> 0 so that for large n, and for all Φ’s except a negligible fraction, the following approximate sparse solution property of Φ holds: For every y having an approximation �y − Φα0�2 ≤ ɛ by a coefficient vector α0 ∈ R m with fewer than ρ · n nonzeros, we have �ˆα1,ɛ − α0�2 ≤ C · ɛ. This has two implications. First: for most Φ, whenever the combinatorial optimization result α0,ɛ would be very sparse, ˆα1,ɛ is a good approximation to α0,ɛ. Second: suppose we are given noisy data obeying y = Φα0 + z where the unknown α0 is known to be sparse and the noise �z�2 ≤ ɛ. For most Φ, noisetolerant ℓ 1minimization will stably recover α0 from y in the presence of noise z. We study also the barelydetermined case m = n and reach parallel conclusions by slightly different arguments. The techniques include the use of almostspherical sections in Banach space theory and concentration of measure for eigenvalues of random matrices.
Fast solution of ℓ1norm minimization problems when the solution may be sparse
, 2006
"... The minimum ℓ1norm solution to an underdetermined system of linear equations y = Ax, is often, remarkably, also the sparsest solution to that system. This sparsityseeking property is of interest in signal processing and information transmission. However, generalpurpose optimizers are much too slo ..."
Abstract

Cited by 47 (1 self)
 Add to MetaCart
The minimum ℓ1norm solution to an underdetermined system of linear equations y = Ax, is often, remarkably, also the sparsest solution to that system. This sparsityseeking property is of interest in signal processing and information transmission. However, generalpurpose optimizers are much too slow for ℓ1 minimization in many largescale applications. The Homotopy method was originally proposed by Osborne et al. for solving noisy overdetermined ℓ1penalized least squares problems. We here apply it to solve the noiseless underdetermined ℓ1minimization problem min ‖x‖1 subject to y = Ax. We show that Homotopy runs much more rapidly than generalpurpose LP solvers when sufficient sparsity is present. Indeed, the method often has the following kstep solution property: if the underlying solution has only k nonzeros, the Homotopy method reaches that solution in only k iterative steps. When this property holds and k is small compared to the problem size, this means that ℓ1 minimization problems with ksparse solutions can be solved in a fraction of the cost of solving one fullsized linear system. We demonstrate this kstep solution property for two kinds of problem suites. First,
Reconstruction and subgaussian operators in Asymptotic Geometric Analysis
 FUNCT. ANAL
"... We present a randomized method to approximate any vector v from some set T ⊂ R n. The data one is given is the set T, vectors (Xi) k i=1 of R n and k scalar products (〈Xi, v〉) k i=1, where (Xi) k i=1 are i.i.d. isotropic subgaussian random vectors in R n, and k ≪ n. We show that with high probabilit ..."
Abstract

Cited by 35 (5 self)
 Add to MetaCart
We present a randomized method to approximate any vector v from some set T ⊂ R n. The data one is given is the set T, vectors (Xi) k i=1 of R n and k scalar products (〈Xi, v〉) k i=1, where (Xi) k i=1 are i.i.d. isotropic subgaussian random vectors in R n, and k ≪ n. We show that with high probability, any y ∈ T for which (〈Xi, y〉) k i=1 is close to the data vector (〈Xi, v〉) k i=1 will be a good approximation of v, and that the degree of approximation is determined by a natural geometric parameter associated with the set T. We also investigate a random method to identify exactly any vector which has a relatively short support using linear subgaussian measurements as above. It turns out that our analysis, when applied to {−1, 1}valued vectors with i.i.d, symmetric entries, yields new information on the geometry of faces of random {−1, 1}polytope; we show that a kdimensional random {−1, 1}polytope with n vertices is mneighborly for very large m ≤ ck / log(c ′ n/k). The proofs are � based on new estimates on the behavior of the empirical process supf∈F �k−1 �k i=1 f 2 (Xi) − Ef 2 � when F is a subset of the L2 sphere. The estimates are given in terms of the γ2 functional with respect to the ψ2 metric on F, and hold both in exponential probability and in expectation.
Temlyakov, A remark on compressed sensing
, 2007
"... Recently, Compressed Sensing (Compressive Sampling) has attracted a lot of attention of both mathematicians and computer scientists. Compressed Sensing refers to a problem of economical recovery of an unknown vector u ∈ R m from the information provided by linear measurements 〈u, ϕj〉, ϕj ∈ R m, j = ..."
Abstract

Cited by 19 (0 self)
 Add to MetaCart
Recently, Compressed Sensing (Compressive Sampling) has attracted a lot of attention of both mathematicians and computer scientists. Compressed Sensing refers to a problem of economical recovery of an unknown vector u ∈ R m from the information provided by linear measurements 〈u, ϕj〉, ϕj ∈ R m, j = 1,..., n. The goal is to design an algorithm
LowDimensional Models for Dimensionality Reduction and Signal Recovery: A Geometric Perspective
, 2009
"... We compare and contrast from a geometric perspective a number of lowdimensional signal models that support stable informationpreserving dimensionality reduction. We consider sparse and compressible signal models for deterministic and random signals, structured sparse and compressible signal model ..."
Abstract

Cited by 18 (10 self)
 Add to MetaCart
We compare and contrast from a geometric perspective a number of lowdimensional signal models that support stable informationpreserving dimensionality reduction. We consider sparse and compressible signal models for deterministic and random signals, structured sparse and compressible signal models, point clouds, and manifold signal models. Each model has a particular geometrical structure that enables signal information in to be stably preserved via a simple linear and nonadaptive projection to a much lower dimensional space whose dimension either is independent of the ambient dimension at best or grows logarithmically with it at worst. As a bonus, we point out a common misconception related to probabilistic compressible signal models, that is, that the generalized Gaussian and Laplacian random models do not support stable linear dimensionality reduction.
Dense error correction via ℓ1 minimization
, 2009
"... This paper studies the problem of recovering a nonnegative sparse signal x ∈ Rn from highly corrupted linear measurements y = Ax + e ∈ Rm, where e is an unknown error vector whose nonzero entries may be unbounded. Motivated by an observation from face recognition in computer vision, this paper prov ..."
Abstract

Cited by 17 (5 self)
 Add to MetaCart
This paper studies the problem of recovering a nonnegative sparse signal x ∈ Rn from highly corrupted linear measurements y = Ax + e ∈ Rm, where e is an unknown error vector whose nonzero entries may be unbounded. Motivated by an observation from face recognition in computer vision, this paper proves that for highly correlated (and possibly overcomplete) dictionaries A, any nonnegative, sufficiently sparse signal x can be recovered by solving an ℓ1minimization problem: min ‖x‖1 + ‖e‖1 subject to y = Ax + e. More precisely, if the fraction ρ of errors is bounded away from one and the support of x grows sublinearly in the dimension m of the observation, then as m goes to infinity, the above ℓ1minimization succeeds for all signals x and almost all signandsupport patterns of e. This result suggests that accurate recovery of sparse signals is possible and computationally feasible even with nearly 100 % of the observations corrupted. The proof relies on a careful characterization of the faces of a convex polytope spanned together by the standard crosspolytope and a set of iid Gaussian vectors with nonzero mean and small variance, which we call the “crossandbouquet ” model. Simulations and experimental results corroborate the findings, and suggest extensions to the result.
Sparse signal reconstruction via iterative support detection
 Siam Journal on Imaging Sciences, issue
, 2010
"... Abstract. We present a novel sparse signal reconstruction method, iterative support detection (ISD), aiming to achieve fast reconstruction and a reduced requirement on the number of measurements compared to the classical ℓ1 minimization approach. ISD addresses failed reconstructions of ℓ1 minimizati ..."
Abstract

Cited by 12 (4 self)
 Add to MetaCart
Abstract. We present a novel sparse signal reconstruction method, iterative support detection (ISD), aiming to achieve fast reconstruction and a reduced requirement on the number of measurements compared to the classical ℓ1 minimization approach. ISD addresses failed reconstructions of ℓ1 minimization due to insufficient measurements. It estimates a support set I from a current reconstruction and obtains a new reconstruction by solving the minimization problem min { ∑ i/∈I xi  : Ax = b}, and it iterates these two steps for a small number of times. ISD differs from the orthogonal matching pursuit method, as well as its variants, because (i) the index set I in ISD is not necessarily nested or increasing, and (ii) the minimization problem above updates all the components of x at the same time. We generalize the null space property to the truncated null space property and present our analysis of ISD based on the latter. We introduce an efficient implementation of ISD, called thresholdISD, for recovering signals with fast decaying distributions of nonzeros from compressive sensing measurements. Numerical experiments show that thresholdISD has significant advantages over the classical ℓ1 minimization approach, as well as two stateoftheart algorithms: the iterative reweighted ℓ1 minimization algorithm (IRL1) and the iterative reweighted leastsquares algorithm (IRLS). MATLAB code is available for download from
Subspaces and orthogonal decompositions generated by bounded orthogonal systems
"... We investigate properties of subspaces of L2 spanned by subsets of a finite orthonormal system bounded in the L ∞ norm. We first prove that there exists an arbitrarily large subset of this orthonormal system on which the L1 and the L2 norms are close, up to a logarithmic factor. Considering for exa ..."
Abstract

Cited by 11 (4 self)
 Add to MetaCart
We investigate properties of subspaces of L2 spanned by subsets of a finite orthonormal system bounded in the L ∞ norm. We first prove that there exists an arbitrarily large subset of this orthonormal system on which the L1 and the L2 norms are close, up to a logarithmic factor. Considering for example the Walsh system, we deduce the existence of two orthogonal subspaces of Ln 2, complementary to each other and each of dimension roughly n/2, spanned by ±1 vectors (i.e. Kashin’s splitting) and in logarithmic distance to the Euclidean space. The same method applies for p> 2, and, in connection with the Λp problem (solved by Bourgain), we study large subsets of this orthonormal system on which the L2 and the Lp norms are close (again, up to a logarithmic factor).