Abstract

Abstract. This article presents new results on using a greedy algorithm, Orthogonal Matching Pursuit (OMP), to solve the sparse approximation problem over redundant dictionaries. It contains a single sufficient condition under which both OMP and Donoho’s Basis Pursuit paradigm (BP) can recover an exactly sparse signal. It leverages this theory to show that both OMP and BP can recover all exactly sparse signals from a wide class of dictionaries. These quasi-incoherent dictionaries offer a natural generalization of incoherent dictionaries, and the Babel function is introduced to quantify the level of incoherence. Indeed, this analysis unifies all the recent results on BP and extends them to OMP. Furthermore, the paper develops a sufficient condition under which OMP can retrieve the common atoms from all optimal representations of a nonsparse signal. From there, it argues that Orthogonal Matching Pursuit is an approximation algorithm for the sparse problem over a quasiincoherent dictionary. That is, for every input signal, OMP can calculate a sparse approximant whose error is only a small factor worse than the optimal error which can be attained with the same number of terms. 1.

Citations

3202	Matrix analysis - Horn, Johnson - 1985
1089	Atomic decomposition by basis pursuit - Chen, Donoho, et al. - 1998
699	Matching Pursuits with Time-Frequency Dictionaries - 27Mallat, Zhang - 1993
478	Constructive Approximation - Devore, Lorentz - 1993
416	Entropy-based algorithms for best-basis selection - 13Coifman, Wickerhauser - 1992
327	Projection pursuit regression - Friedman, Stuetzle - 1981
244	Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization - Donoho, Elad
243	Uncertainty principles and ideal atomic decomposition - Donoho, Huo - 2001
192	Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition - Pati, Rezaiifar, et al. - 1993
184	Foundations of time-frequency analysis - Gröchenig - 2001
154	Introductory functional analysis with applications - Kreyszig - 1989
152	Sparse representations in unions of bases - Gribonval, Nielsen - 2003
142	A generalized uncertainty principle and sparse representation in pairs of bases - Elad, Bruckstein - 1997
111	Jr., “Grassmannian frames with applications to coding and communication - Strohmer, Heath - 2003
102	Orthogonal least squares methods and their application to non-linear system identification - Chen, Billings, et al. - 1989
60	Block coordinate relaxation methods for nonparametric wavelet denoising - Sardy, Bruce, et al. - 2000
58	Approximation of functions over redundant dictionaries using coherence, in: the fourteenth annual ACM-SIAM symposium on Discrete algorithms - Gilbert, Muthukrishnan, et al. - 2003
56	Adaptive time-frequency decompositions - Mallat, Davis, et al. - 1994
52	High-Dimensional Computational Geometry - Indyk - 2000
43	On a conjecture of huber concerning the convergence of projection pursuit regression - Jones - 1987
40	Nonlinear methods of approximation - Temlyakov
39	Greedy adaptive approximation - Davis, Mallat, et al. - 1997
24	Signal representation using adaptive normalized Gaussian functions - Qian, Chen - 1994
21	Improved sparse approximation over quasi-incoherent dictionaries - Tropp, Gilbert, et al. - 2003
12	On quasi-orthogonal signatures for CDMA systems - Heath, Strohmer, et al.
5	On sparse representations in pairs of bases, Accepted to the - Feuer, Nemirovsky
4	approximation with walsh atoms - Best - 1997
1	Existence of real Grassmannian frames - Sustik, Tropp - 2003