Results 1 
5 of
5
Greed is Good: Algorithmic Results for Sparse Approximation
, 2004
"... This article presents new results on using a greedy algorithm, orthogonal matching pursuit (OMP), to solve the sparse approximation problem over redundant dictionaries. It provides a sufficient condition under which both OMP and Donoho’s basis pursuit (BP) paradigm can recover the optimal representa ..."
Abstract

Cited by 547 (6 self)
 Add to MetaCart
This article presents new results on using a greedy algorithm, orthogonal matching pursuit (OMP), to solve the sparse approximation problem over redundant dictionaries. It provides a sufficient condition under which both OMP and Donoho’s basis pursuit (BP) paradigm can recover the optimal representation of an exactly sparse signal. It leverages this theory to show that both OMP and BP succeed for every sparse input signal from a wide class of dictionaries. These quasiincoherent dictionaries offer a natural generalization of incoherent dictionaries, and the cumulative coherence function is introduced to quantify the level of incoherence. This analysis unifies all the recent results on BP and extends them to OMP. Furthermore, the paper develops a sufficient condition under which OMP can identify atoms from an optimal approximation of a nonsparse signal. From there, it argues that OMP is an approximation algorithm for the sparse problem over a quasiincoherent dictionary. That is, for every input signal, OMP calculates a sparse approximant whose error is only a small factor worse than the minimal error that can be attained with the same number of terms.
Elad M 2003 Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ 1 minimization
 Proc. Natl Acad. Sci. USA 100 2197–202
"... Given a ‘dictionary ’ D = {dk} of vectors dk, we seek to represent a signal S as a linear combination S = ∑ k γ(k)dk, with scalar coefficients γ(k). In particular, we aim for the sparsest representation possible. In general, this requires a combinatorial optimization process. Previous work considere ..."
Abstract

Cited by 382 (32 self)
 Add to MetaCart
Given a ‘dictionary ’ D = {dk} of vectors dk, we seek to represent a signal S as a linear combination S = ∑ k γ(k)dk, with scalar coefficients γ(k). In particular, we aim for the sparsest representation possible. In general, this requires a combinatorial optimization process. Previous work considered the special case where D is an overcomplete system consisting of exactly two orthobases, and has shown that, under a condition of mutual incoherence of the two bases, and assuming that S has a sufficiently sparse representation, this representation is unique and can be found by solving a convex optimization problem: specifically, minimizing the ℓ1 norm of the coefficients γ. In this paper, we obtain parallel results in a more general setting, where the dictionary D can arise from two or several bases, frames, or even less structured systems. We introduce the Spark, ameasure of linear dependence in such a system; it is the size of the smallest linearly dependent subset (dk). We show that, when the signal S has a representation using less than Spark(D)/2 nonzeros, this representation is necessarily unique.
Quantitative Robust Uncertainty Principles and Optimally Sparse Decompositions
, 2004
"... In this paper, we develop a robust uncertainty principle for finite signals in C N which states that for nearly all choices T, Ω ⊂ {0,..., N − 1} such that T  + Ω  ≍ (log N) −1/2 · N, there is no signal f supported on T whose discrete Fourier transform ˆ f is supported on Ω. In fact, we can mak ..."
Abstract

Cited by 123 (13 self)
 Add to MetaCart
In this paper, we develop a robust uncertainty principle for finite signals in C N which states that for nearly all choices T, Ω ⊂ {0,..., N − 1} such that T  + Ω  ≍ (log N) −1/2 · N, there is no signal f supported on T whose discrete Fourier transform ˆ f is supported on Ω. In fact, we can make the above uncertainty principle quantitative in the sense that if f is supported on T, then only a small percentage of the energy (less than half, say) of ˆ f is concentrated on Ω. As an application of this robust uncertainty principle (QRUP), we consider the problem of decomposing a signal into a sparse superposition of spikes and complex sinusoids f(s) = � α1(t)δ(s − t) + � α2(ω)e i2πωs/N / √ N. t∈T We show that if a generic signal f has a decomposition (α1, α2) using spike and frequency locations in T and Ω respectively, and obeying ω∈Ω T  + Ω  ≤ Const · (log N) −1/2 · N, then (α1, α2) is the unique sparsest possible decomposition (all other decompositions have more nonzero terms). In addition, if T  + Ω  ≤ Const · (log N) −1 · N, then the sparsest (α1, α2) can be found by solving a convex optimization problem. Underlying our results is a new probabilistic approach which insists on finding the correct uncertainty relation or the optimally sparse solution for nearly all subsets but not necessarily all of them, and allows to considerably sharpen previously known results [9, 10]. In fact, we show that the fraction of sets (T, Ω) for which the above properties do not hold can be upper bounded by quantities like N −α for large values of α. The QRUP (and the application to finding sparse representations) can be extended to general pairs of orthogonal bases Φ1, Φ2 of C N. For nearly all choices Γ1, Γ2 ⊂ {0,..., N − 1} obeying Γ1  + Γ2  ≍ µ(Φ1, Φ2) −2 · (log N) −m, where m ≤ 6, there is no signal f such that Φ1f is supported on Γ1 and Φ2f is supported on Γ2 where µ(Φ1, Φ2) is the mutual coherence between Φ1 and Φ2.
Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information
, 2004
"... This paper considers the model problem of reconstructing an object from incomplete frequency samples. Consider a discretetime signal f ∈ C N and a randomly chosen set of frequencies Ω. Is it possible to reconstruct f from the partial knowledge of its Fourier coefficients on the set Ω? A typical res ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
This paper considers the model problem of reconstructing an object from incomplete frequency samples. Consider a discretetime signal f ∈ C N and a randomly chosen set of frequencies Ω. Is it possible to reconstruct f from the partial knowledge of its Fourier coefficients on the set Ω? A typical result of this paper is as follows. Suppose that f is a superposition of T  spikes f(t) = ∑ τ∈T f(τ) δ(t − τ) obeying T  ≤ CM · (log N) −1 · Ω, for some constant CM> 0. We do not know the locations of the spikes nor their amplitudes. Then with probability at least 1−O(N −M), f can be reconstructed exactly as the solution to the ℓ1 minimization problem g(t), s.t. ˆg(ω) = ˆ f(ω) for all ω ∈ Ω.
Quantitative Robust Uncertainty Principles and Optimally Sparse Decompositions
, 2005
"... In this paper, we develop a robust uncertainty principle for finite signals in C N which states that for nearly all choices T, Ω ⊂ {0,..., N − 1} such that T  + Ω  ≃ (log N) −1/2 · N, there is no signal f supported on T whose discrete Fourier transform ˆ f is supported on Ω. In fact, we can mak ..."
Abstract
 Add to MetaCart
In this paper, we develop a robust uncertainty principle for finite signals in C N which states that for nearly all choices T, Ω ⊂ {0,..., N − 1} such that T  + Ω  ≃ (log N) −1/2 · N, there is no signal f supported on T whose discrete Fourier transform ˆ f is supported on Ω. In fact, we can make the above uncertainty principle quantitative in the sense that if f is supported on T, then only a small percentage of the energy (less than half, say) of ˆ f is concentrated on Ω. As an application of this robust uncertainty principle (QRUP), we consider the problem of decomposing a signal into a sparse superposition of spikes and complex sinusoids f(s) = ∑ α1(t)δ(s − t) + α2(ω)e i2πωs/N / √ N. t∈T