## Iterative hard thresholding for compressed sensing

### Cached

### Download Links

- [www.see.ed.ac.uk]
- [www.personal.soton.ac.uk]
- [users.fmrib.ox.ac.uk]
- [arxiv.org]
- [www.see.ed.ac.uk]
- DBLP

### Other Repositories/Bibliography

Venue: | Appl. Comp. Harm. Anal |

Citations: | 138 - 13 self |

### BibTeX

@ARTICLE{Blumensath_iterativehard,

author = {Thomas Blumensath and Mike E. Davies},

title = {Iterative hard thresholding for compressed sensing},

journal = {Appl. Comp. Harm. Anal},

year = {}

}

### Years of Citing Articles

### OpenURL

### Abstract

Compressed sensing is a technique to sample compressible signals below the Nyquist rate, whilst still allowing near optimal reconstruction of the signal. In this paper we present a theoretical analysis of the iterative hard thresholding algorithm when applied to the compressed sensing recovery problem. We show that the algorithm has the following properties (made more precise in the main text of the paper) • It gives near-optimal error guarantees. • It is robust to observation noise. • It succeeds with a minimum number of observations. • It can be used with any sampling operator for which the operator and its adjoint can be computed. • The memory requirement is linear in the problem size. Preprint submitted to Elsevier 28 January 2009 • Its computational complexity per iteration is of the same order as the application of the measurement operator or its adjoint. • It requires a fixed number of iterations depending only on the logarithm of a form of signal to noise ratio of the signal. • Its performance guarantees are uniform in that they only depend on properties of the sampling operator and signal sparsity.

### Citations

6083 |
A mathematical theory of communication
- Shannon
- 1948
(Show Context)
Citation Context ...Algorithms, Compressed Sensing, Sparse Inverse Problem, Signal Recovery, Iterative Hard Thresholding PACS: Reconstruction algorithms 1 Introduction For more than fifty years, the Nyquist-Shannon [16] =-=[18]-=- sampling theorem was generally used as the foundation of signal acquisition systems. Using this theory, it was a commonly held belief that signals have to be sampled at twice the signal bandwidth. Wh... |

4697 | Topics in Matrix Analysis - Horn, Johnson - 1991 |

1746 | Compressed sensing
- Donoho
- 2006
(Show Context)
Citation Context ...for general band-limited signals, this theory does not account for additional signal structures that might be known a priori. The recently emerging field of compressed sensing, [8], [4], [5], [3] and =-=[7]-=- and the related theory of signals with a finite rate of innovations [20], 1 This research was supported by EPSRC grants D000246/2 and EP/F039697/1. MED acknowledges support of his position from the S... |

1318 | Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information
- Romberg, Tao
- 2006
(Show Context)
Citation Context ...ilst this is true for general band-limited signals, this theory does not account for additional signal structures that might be known a priori. The recently emerging field of compressed sensing, [8], =-=[4]-=-, [5], [3] and [7] and the related theory of signals with a finite rate of innovations [20], 1 This research was supported by EPSRC grants D000246/2 and EP/F039697/1. MED acknowledges support of his p... |

751 |
Stable Signal recovery from incomplete and inaccurate measurements
- Candès, Romberg, et al.
(Show Context)
Citation Context ...this is true for general band-limited signals, this theory does not account for additional signal structures that might be known a priori. The recently emerging field of compressed sensing, [8], [4], =-=[5]-=-, [3] and [7] and the related theory of signals with a finite rate of innovations [20], 1 This research was supported by EPSRC grants D000246/2 and EP/F039697/1. MED acknowledges support of his positi... |

350 | CoSaMP: Iterative signal recovery from incomplete and inaccurate samples
- Needell, Tropp
- 2008
(Show Context)
Citation Context ...g in [19]. Better theoretical properties were recently proven for a regularised Orthogonal Matching Pursuit algorithm [15] [14]. Even more recently, the Compressive Sampling Matching Pursuit (CoSaMP) =-=[13]-=- and the nearly identical Subspace Pursuit [6] algorithms were introduced and analysed for compressed sensing signal reconstruction. Of all of these methods, CoSaMP currently offers the most comprehen... |

218 | Sampling signals with finite rate of innovation
- Vetterli, Marziliano, et al.
(Show Context)
Citation Context ...ional signal structures that might be known a priory. The recently emerging field of compressed sensing, [3], [4], [5], [6] and [7] and the related theory of signals with a finite rate of innovations =-=[8]-=-, start from another premise. In compressed sensing, signals are assumed to be sparse in some transform domain. This sparsity constraint significantly reduces the size of the set of possible signals c... |

171 |
Certain topics in telegraph transmission theory
- NYQUIST
- 1928
(Show Context)
Citation Context ...rds: Algorithms, Compressed Sensing, Sparse Inverse Problem, Signal Recovery, Iterative Hard Thresholding PACS: Reconstruction algorithms 1 Introduction For more than fifty years, the Nyquist-Shannon =-=[16]-=- [18] sampling theorem was generally used as the foundation of signal acquisition systems. Using this theory, it was a commonly held belief that signals have to be sampled at twice the signal bandwidt... |

147 | Signal recovery from partial information via orthogonal matching pursuit
- Tropp, Gilbert
(Show Context)
Citation Context ... from compressed sensing observations are greedy methods. A by now traditional approach is Orthogonal Matching Pursuit [10], which was analysed as a reconstruction algorithm for compressed sensing in =-=[19]-=-. Better theoretical properties were recently proven for a regularised Orthogonal Matching Pursuit algorithm [15] [14]. Even more recently, the Compressive Sampling Matching Pursuit (CoSaMP) [13] and ... |

122 | Quantitative robust uncertainty principles and optimally sparse decompositions
- Candès, Romberg
- 2006
(Show Context)
Citation Context ...is true for general band-limited signals, this theory does not account for additional signal structures that might be known a priori. The recently emerging field of compressed sensing, [8], [4], [5], =-=[3]-=- and [7] and the related theory of signals with a finite rate of innovations [20], 1 This research was supported by EPSRC grants D000246/2 and EP/F039697/1. MED acknowledges support of his position fr... |

103 | Uniform uncertainty principle and signal recovery via regularized orthogonal matching pursuit
- Needell, Vershynin
- 2008
(Show Context)
Citation Context ...rsuit [10], which was analysed as a reconstruction algorithm for compressed sensing in [19]. Better theoretical properties were recently proven for a regularised Orthogonal Matching Pursuit algorithm =-=[15]-=- [14]. Even more recently, the Compressive Sampling Matching Pursuit (CoSaMP) [13] and the nearly identical Subspace Pursuit [6] algorithms were introduced and analysed for compressed sensing signal r... |

79 | Sparse reconstruction by convex relaxation: Fourier and Gaussian measurements, preprint
- Rudelson, Vershynin
(Show Context)
Citation Context ...h as the restricted isometry property of the next subsection), which allow for an efficient estimation of x. The interested reader is referred to the extensive literature, such as [4], [5], [3], [7], =-=[17]-=-, [11], [12], [13] and references therein. The second problem, which is the focus of this paper, is the study of concrete algorithms for the efficient estimation of x given only y and Φ. 2.2 The Restr... |

75 |
Adaptive time-frequency decomposition
- Davis, Mallat, et al.
- 1994
(Show Context)
Citation Context ...nother set of algorithms, which could be shown to efficiently reconstruct signals from compressed sensing observations are greedy methods. A by now traditional approach is Orthogonal Matching Pursuit =-=[9]-=-, which was analysed as a reconstruction algorithm for compressed sensing in [10]. Better theoretical properties were recently The authors are with Institute for Digital Communications & the Joint Res... |

73 |
Neuromagnetic source imaging with focuss: a recursive weighted minimum norm algorithm, Electroencephalography and Clinical Neurophysiology 95
- Gorodnitsky, George, et al.
- 1995
(Show Context)
Citation Context ...h. Whilst this is true for general band-limited signals, this theory does not account for additional signal structures that might be known a priori. The recently emerging field of compressed sensing, =-=[8]-=-, [4], [5], [3] and [7] and the related theory of signals with a finite rate of innovations [20], 1 This research was supported by EPSRC grants D000246/2 and EP/F039697/1. MED acknowledges support of ... |

66 |
The widths of a Euclidean ball
- Garnaev, Gluskin
- 1984
(Show Context)
Citation Context ...e approximated with s-sparse signals. To show that this result is in fact optimal up to a constant, we can use the following argument. Theoretic considerations due to Kashin [17] and Garnaeva-Gluskin =-=[18]-=- show that any matrix Φ ∈ R M×N must have at least M ≥ cs log(N/s) for some constant c in order for the observation x = Φy to allow a reconstruction ˆy with ‖y − ˆy‖2 ≤ C/ √ s‖y‖1. As discussed above,... |

63 |
Iterative thresholding for sparse approximations,” The
- Blumensath, Davies
- 2008
(Show Context)
Citation Context ...ot on the size of the non-zero signal coefficients. Furthermore, it requires minimal storage and computations and works with (up to a constant) a minimal number of observations. 3In a previous paper =-=[1]-=-, iterative hard thresholding algorithms were studied. In particular, their convergence to fixed points of ℓ0 regularised (or constrained) cost functions could be proven. In this paper, it is shown th... |

60 | Subspace pursuit for compressive sensing: Closing the gap between performance and complexity. arXiv:0803.0811v1 [cs.NA
- Dai, Milenkovic
- 2008
(Show Context)
Citation Context ...ecently proven for a regularised Orthogonal Matching Pursuit algorithm [15] [14]. Even more recently, the Compressive Sampling Matching Pursuit (CoSaMP) [13] and the nearly identical Subspace Pursuit =-=[6]-=- algorithms were introduced and analysed for compressed sensing signal reconstruction. Of all of these methods, CoSaMP currently offers the most comprehensive set of theoretic performance guarantees. ... |

48 | Signal recovery from incomplete and inaccurate measurements via regularized orthogonal matching pursuit
- Needell, Vershynin
(Show Context)
Citation Context ...nburgh EH9 3JL, UK (Tel.: +44(0)131 6513492, Fax.: +44(0)131 6506554, e-mail: thomas.blumensath@ed.ac.uk, mike.davies@ed.ac.uk).2 proven for a regularised Orthogonal Matching Pursuit algorithm [11], =-=[12]-=-. Even more recently, the Subspace Pursuit [13] and the nearly identical Compressive Sampling Matching Pursuit (CoSaMP) [14] algorithms were introduced and analysed for compressed sensing signal recon... |

35 | Reconstruction and subgaussian operators in asymptotic geometric analysis
- Mendelson, Pajor, et al.
- 2006
(Show Context)
Citation Context ...tricted isometry property of the next subsection), which allow for an efficient estimation of x. The interested reader is referred to the extensive literature, such as [4], [5], [3], [7], [17], [11], =-=[12]-=-, [13] and references therein. The second problem, which is the focus of this paper, is the study of concrete algorithms for the efficient estimation of x given only y and Φ. 2.2 The Restricted Isomet... |

25 |
The widths of certain finite dimensional sets and classes of smooth functions
- Kashin
- 1977
(Show Context)
Citation Context ...imited by how well y can be approximated with s-sparse signals. To show that this result is in fact optimal up to a constant, we can use the following argument. Theoretic considerations due to Kashin =-=[17]-=- and Garnaeva-Gluskin [18] show that any matrix Φ ∈ R M×N must have at least M ≥ cs log(N/s) for some constant c in order for the observation x = Φy to allow a reconstruction ˆy with ‖y − ˆy‖2 ≤ C/ √ ... |

11 |
Iterative image coding with overcomplete complex wavelet transforms
- Kingsbury, Reeves
- 1253
(Show Context)
Citation Context ...= |Γ ⋃ Λ| ‖Φ T Γ ΦΛxΛ‖2 ≤ δs‖xΛ‖2. (6) 3 Iterative Hard Thresholding 3.1 Definition of the Algorithm In [1], we discussed the following Iterative Hard Thresholding algorithm (IHTs) previously used in =-=[9]-=-. Let x [0] = 0 and use the iteration x [n+1] = Hs(x [n] + Φ T (y − Φx [n] )), (7) where Hs(a) is the non-linear operator that sets all but the largest (in magnitude) s elements of a to zero. If there... |

5 |
A simple, efficient and near optimal algorithm for compressed sensing
- Blumensath, Davies
- 2009
(Show Context)
Citation Context ...onent part of the Edinburgh Research Partnership. An abridged earlier version of this manuscript has been submitted to the International Conference on Acoustics, Speech and Signal Processing (ICASSP) =-=[2]-=-. 2start from another premise. In compressed sensing, signals are assumed to be (approximately) sparse in some transform domain. This sparsity constraint significantly reduces the size of the set of ... |

4 |
Iterative thresholding for sparse approximations, to appear in
- Blumensath, Davies
(Show Context)
Citation Context ...state of the art algorithms such as CoSaMP and has recovery guarantees of the same order as ℓ1 based approaches. At a first glance, this seems to be at odds with previously reported numerical results =-=[15]-=-, which have shown that IHTs does not perform as well as other methods such as Orthogonal Matching Pursuit, for which there are currently no comparable performance guarantees. What is more, IHTs does ... |

2 |
Tomczak Jaegermann, N.: Uniform uncertainty principle for Bernoulli and subgaussian ensembles
- Mendelson, Pajor
- 2009
(Show Context)
Citation Context ...he restricted isometry property of the next subsection), which allow for an efficient estimation of x. The interested reader is referred to the extensive literature, such as [4], [5], [3], [7], [17], =-=[11]-=-, [12], [13] and references therein. The second problem, which is the focus of this paper, is the study of concrete algorithms for the efficient estimation of x given only y and Φ. 2.2 The Restricted ... |

2 |
COSAMP: Iterative signal reovery from incomplete and inacurate samples.,” submitted
- Needell, Tropp
- 2008
(Show Context)
Citation Context ...uk).2 proven for a regularised Orthogonal Matching Pursuit algorithm [11], [12]. Even more recently, the Subspace Pursuit [13] and the nearly identical Compressive Sampling Matching Pursuit (CoSaMP) =-=[14]-=- algorithms were introduced and analysed for compressed sensing signal reconstruction. Of all of these methods, CoSaMP currently offers the most comprehensive set of theoretic performance guarantees. ... |