## Exploiting structure in wavelet-based Bayesian compressive sensing (2009)

Citations: | 49 - 9 self |

### BibTeX

@MISC{He09exploitingstructure,

author = {Lihan He and Lawrence Carin},

title = {Exploiting structure in wavelet-based Bayesian compressive sensing },

year = {2009}

}

### OpenURL

### Abstract

Bayesian compressive sensing (CS) is considered for signals and images that are sparse in a wavelet basis. The statistical structure of the wavelet coefficients is exploited explicitly in the proposed model, and therefore this framework goes beyond simply assuming that the data are compressible in a wavelet basis. The structure exploited within the wavelet coefficients is consistent with that used in waveletbased compression algorithms. A hierarchical Bayesian model is constituted, with efficient inference via Markov chain Monte Carlo (MCMC) sampling. The algorithm is fully developed and demonstrated using several natural images, with performance comparisons to many state-of-the-art compressive-sensing inversion algorithms.

### Citations

1864 | Compressed sensing
- Donoho
(Show Context)
Citation Context ...ectly, thereby reducing measurement costs, while still retaining all of the informative parts of the data? This goal has spawned the new field of compressive sensing (or compressed sensing) [6], [7], =-=[8]-=-, in which it has been demonstrated that if the signal of interest is sparse in some basis, then with a relatively small number of appropriately designed projection measurements the underlying signal ... |

1777 | Atomic decomposition by basis pursuit - Chen, Donoho, et al. - 1998 |

1401 | Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
- Candès, Romberg, et al.
(Show Context)
Citation Context ...a directly, thereby reducing measurement costs, while still retaining all of the informative parts of the data? This goal has spawned the new field of compressive sensing (or compressed sensing) [6], =-=[7]-=-, [8], in which it has been demonstrated that if the signal of interest is sparse in some basis, then with a relatively small number of appropriately designed projection measurements the underlying si... |

1312 | A tutorial on particle filters for online nonlinear/nonGaussian Bayesian tracking
- Arulampalam, Maskell, et al.
- 2002
(Show Context)
Citation Context ...ults presented below. The use of approximate (truncated) MCMC sampling is distinct from but related to particle filters, which have recently seen significant utility in signal-processing applications =-=[30]-=-. 0.7 Relative Reconstruction Error in Wavelet Domain 0.65 0.6 0.55 0.5 0.45 0.4 0.35 0.3 0 50 100 150 200 250 300 MCMC Iteration Fig. 2. Example of an MCMC convergence curve for the image cameraman o... |

1309 |
Embedded image coding using zerotrees of wavelet coefficients
- Shapiro
- 1993
(Show Context)
Citation Context ...the-art wavelet-based compression algorithms that are based upon “zero trees” (subtrees of wavelet coefficients that may all be set to zero with negligible effect on the reconstruction accuracy) [4], =-=[37]-=-. The motivation for the HMT construct is discussed in detail in [5]. III. TREE-STRUCTURED WAVELET COMPRESSIVE SENSING A. Compressive Sensing with Wavelet-Transform Coefficients Assume a discrete sign... |

925 | The JPEG still picture compression standard
- Wallace
- 1991
(Show Context)
Citation Context ...ed toward development of transform codes, with the discrete-cosine and wavelet transforms [1] constituting two important examples. The discrete cosine transform (DCT) is employed in the JPEG standard =-=[2]-=-, with wavelets employed in the JPEG2000 standard [3]. Wavelet-based transform coding [4] explicitly exploits the structure [5] manifested in the wavelet coefficients of typical data. Specifically, fo... |

888 | Near-optimal signal recovery from random projections: Universal encoding strategies
- Candès, Tao
(Show Context)
Citation Context ... to imposing and exploiting prior knowledge about structure in a wavelet decomposition of images, other forms of prior knowledge have been exploited in CS. For example the use of total variation (TV) =-=[27]-=- is generally a non-statistical approach that may be employed to account for prior knowledge about the properties of images. Researchers have also developed techniques that impose prior structure thro... |

812 | Least angle regression
- Efron, Hastie, et al.
- 2004
(Show Context)
Citation Context ...is i.i.d. assumption does not impose anticipated structure/correlation between transform coefficients. While this leads to development of many algorithms for CS inversion (see [15], [16], [12], [14], =-=[17]-=-, among many others), such a formulation does not exploit all of the prior information available about the transform coefficients θ. For example, as discussed above with respect to the wavelet transfo... |

690 |
An introduction to compressive sampling
- Candes, Wakin
- 2008
(Show Context)
Citation Context ...ny coefficients are negligibly small, but not exactly zero), one may still employ compressive sensing (CS) to recover the data up to an error proportional to the energy in the negligible coefficients =-=[9]-=-. Two of the early important applications of CS are in MRI [10] and in development of new hyperspectral cameras April 7, 2009 DRAFT3 [11]. Details on how to design the compressive-sensing projection ... |

615 | Sparse Bayesian learning and the relevance vector machine
- Tipping
- 2001
(Show Context)
Citation Context .... As an example of how these parameters were selected, a0, b0, c0 and d0 have been set in a non-informative manner consistent with related regression models, such as the relevance vector machine (see =-=[40]-=-). Concerning setting the parameters on the Beta distributions, our goal is to impose the belief that if a parent coefficient is zero, then it is likely that its children will also be zero. As indicat... |

386 |
efficient image codec based on set partitioning in hierarchical trees
- Said, Pearlman, et al.
- 1996
(Show Context)
Citation Context ... [1] constituting two important examples. The discrete cosine transform (DCT) is employed in the JPEG standard [2], with wavelets employed in the JPEG2000 standard [3]. Wavelet-based transform coding =-=[4]-=- explicitly exploits the structure [5] manifested in the wavelet coefficients of typical data. Specifically, for most natural data (signals and images) the wavelet coefficients are compressible, imply... |

366 | CoSaMP: Iterative signal recovery from incomplete and inaccurate samples
- Needell, Tropp
- 2008
(Show Context)
Citation Context ...le but not exactly sparse. We note that there are many CS inversion algorithms that do not explicitly solve an ℓ1-based inversion but that are similarly motivated by sparseness; among these are [12], =-=[13]-=-, [14]. The aforementioned ℓ1 inversion may be viewed as a maximum a posteriori estimate for θ under the assumption that each component of θ is drawn i.i.d. from a Laplace prior [15]. This i.i.d. assu... |

365 |
Variable Selection via Gibbs Sampling
- George, McCulloch
- 1993
(Show Context)
Citation Context ...oefficients; both the high and low states are characterized by Gaussian statistics. In the model proposed here a different but related construction is employed, in terms of a “spike-slab” prior [32], =-=[33]-=-, [34], [35], [36], and here the coefficients in the “low” states are explicitly set April 7, 2009 DRAFT5 to zero (sparseness is explicitly imposed). The proposed Bayesian approach yields “error bars... |

336 | WaveletBased Statistical Signal Processing Using Hidden Markov Models
- Crouse, Nowak, et al.
- 1998
(Show Context)
Citation Context ...ized in a statistical setting, building on recent research on Bayesian CS [15]. The proposed method utilizes ideas related to the hidden Markov tree statistical representation of wavelet coefficients =-=[5]-=-, and an efficient MCMC inference engine has been constituted. On all examples considered to date, considering real imagery, we have observed very fast convergence of the MCMC algorithm; the inference... |

325 | Model-based compressive sensing
- Baraniuk, Cevher, et al.
- 1982
(Show Context)
Citation Context ...train or regularize the inversion, ideally reducing the number of required CS measurements N. This concept has been made rigorous recently for sparse θ [18], as well as for compressible θ [19], [20], =-=[21]-=-; these papers demonstrate that one may achieve accurate CS inversions with substantially fewer projection measurements (smaller N) if known properties of the structure of April 7, 2009 DRAFT4 θ are ... |

324 | Signal recovery from random measurements via orthogonal matching pursuit
- Tropp, Gilbert
- 2007
(Show Context)
Citation Context ...ressible but not exactly sparse. We note that there are many CS inversion algorithms that do not explicitly solve an ℓ1-based inversion but that are similarly motivated by sparseness; among these are =-=[12]-=-, [13], [14]. The aforementioned ℓ1 inversion may be viewed as a maximum a posteriori estimate for θ under the assumption that each component of θ is drawn i.i.d. from a Laplace prior [15]. This i.i.d... |

304 | A parametric texture model based on joint statistics of complex wavelets coefficients
- Portilla, Simoncelli
- 2000
(Show Context)
Citation Context ...is and processing applications. As indicated above, this structure is employed in image compression [4]. It is also employed in image denoising [22], as well as texture synthesis and image inpainting =-=[23]-=-. More directly related to CS, the wavelet tree structure has been employed in non-statistical CS inversion [24], and within more statistical settings via the hidden Markov tree (HMT) [25]. There have... |

293 | Variational Algorithms for Approximate Bayesian Inference
- Beal
- 2003
(Show Context)
Citation Context ...n MCMC convergence curve for an image of size 128×128. The vertical axis is evaluated as ‖θ− ˆ θ‖2/‖θ‖2, where ˆ θ denotes the reconstructed wavelet coefficients. The variational Bayesian (VB) method =-=[42]-=- is often considered for fast but approximate Bayesian inference. The MCMC solution employed here was found to be relatively accurate and fast, and therefore we did not implement VB inference. Given t... |

244 |
Sparse MRI: The application of compressed sensing for rapid
- Lustig, Donoho, et al.
- 2007
(Show Context)
Citation Context ...ne may still employ compressive sensing (CS) to recover the data up to an error proportional to the energy in the negligible coefficients [9]. Two of the early important applications of CS are in MRI =-=[10]-=- and in development of new hyperspectral cameras April 7, 2009 DRAFT3 [11]. Details on how to design the compressive-sensing projection vectors, and requirements on the (typically relatively small) n... |

229 | The relevance vector machine
- Tipping
- 2000
(Show Context)
Citation Context ...P, SolveStOMP, and SolveLasso, respectively, from the SparseLab toolbox3 . The BCS algorithm can be implemented via the relevance vector machines (RVM) [15]; we implemented it using a variational RVM =-=[43]-=-. All software are written in MATLAB T M , and run on PCs with 3.6GHz CPU and 4GB memory. 1 Code at http : //www.ece.duke.edu/ ∼ shji/BCS.html 2 http : //www.acm.caltech.edu/l1magic/ 3 http : //sparse... |

194 |
A Wavelet Tour
- Mallat
- 1998
(Show Context)
Citation Context ..., sparseness, compression I. INTRODUCTION Over the last two decades there has been significant research directed toward development of transform codes, with the discrete-cosine and wavelet transforms =-=[1]-=- constituting two important examples. The discrete cosine transform (DCT) is employed in the JPEG standard [2], with wavelets employed in the JPEG2000 standard [3]. Wavelet-based transform coding [4] ... |

184 | Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit
- Donoho, Tsaig, et al.
- 2006
(Show Context)
Citation Context ... not exactly sparse. We note that there are many CS inversion algorithms that do not explicitly solve an ℓ1-based inversion but that are similarly motivated by sparseness; among these are [12], [13], =-=[14]-=-. The aforementioned ℓ1 inversion may be viewed as a maximum a posteriori estimate for θ under the assumption that each component of θ is drawn i.i.d. from a Laplace prior [15]. This i.i.d. assumption... |

148 | Bayesian compressive sensing
- Ji, Xue, et al.
(Show Context)
Citation Context ...g these are [12], [13], [14]. The aforementioned ℓ1 inversion may be viewed as a maximum a posteriori estimate for θ under the assumption that each component of θ is drawn i.i.d. from a Laplace prior =-=[15]-=-. This i.i.d. assumption does not impose anticipated structure/correlation between transform coefficients. While this leads to development of many algorithms for CS inversion (see [15], [16], [12], [1... |

144 | Sparsity and incoherence in compressive sampling
- Candès, Romberg
- 2007
(Show Context)
Citation Context ...e data directly, thereby reducing measurement costs, while still retaining all of the informative parts of the data? This goal has spawned the new field of compressive sensing (or compressed sensing) =-=[6]-=-, [7], [8], in which it has been demonstrated that if the signal of interest is sparse in some basis, then with a relatively small number of appropriately designed projection measurements the underlyi... |

143 | Iterative hard thresholding for compressed sensing
- Blumensath, Davies
- 2009
(Show Context)
Citation Context ...ls (structure within the transform coefficients), by introducing dependencies between locations of the signal coefficients. Two greedy CS algorithms, CoSaMP [13] and iterative hard thresholding (IHT) =-=[38]-=-, are implemented in [21], with the wavelet tree structure incorporated into the inversion models. In this paper the proposed tree-structured wavelet compressive sensing (TSW-CS) model is constructed ... |

123 | Bayesian factor regression models in the large p, small n paradigm
- West
- 2003
(Show Context)
Citation Context ...the high and low states are characterized by Gaussian statistics. In the model proposed here a different but related construction is employed, in terms of a “spike-slab” prior [32], [33], [34], [35], =-=[36]-=-, and here the coefficients in the “low” states are explicitly set April 7, 2009 DRAFT5 to zero (sparseness is explicitly imposed). The proposed Bayesian approach yields “error bars” on the recovered... |

108 |
AWavelet Tour of Signal Processing, 2nd edition
- Mallat
- 1999
(Show Context)
Citation Context ..., sparseness, compression I. INTRODUCTION Over the last two decades there has been significant research directed toward development of transform codes, with the discrete-cosine and wavelet transforms =-=[1]-=- constituting two important examples. The discrete cosine transform (DCT) is employed in the JPEG standard [2], with wavelets employed in the JPEG2000 standard [3]. Wavelet-based transform coding [4] ... |

90 | Distributed compressed sensing
- Baron, Wakin, et al.
- 2005
(Show Context)
Citation Context ...that do not exploit the structure inherent to the wavelet coefficients. Concerning future research, there has recently been interest in the simultaneous inversion of multiple distinct CS measurements =-=[44]-=-, [45] (by sharing information between these different measurements, the total number of CS measurements may be reduced). The Bayesian setting proposed here is particularly amenable to the joint proce... |

60 | Bayesian variable selection with related predictors
- Chipman
- 1996
(Show Context)
Citation Context ...ients; both the high and low states are characterized by Gaussian statistics. In the model proposed here a different but related construction is employed, in terms of a “spike-slab” prior [32], [33], =-=[34]-=-, [35], [36], and here the coefficients in the “low” states are explicitly set April 7, 2009 DRAFT5 to zero (sparseness is explicitly imposed). The proposed Bayesian approach yields “error bars” on t... |

58 |
High-dimensional sparse factor modeling: applications in gene expression genomics
- Carvalho, Chang, et al.
(Show Context)
Citation Context ... both the high and low states are characterized by Gaussian statistics. In the model proposed here a different but related construction is employed, in terms of a “spike-slab” prior [32], [33], [34], =-=[35]-=-, [36], and here the coefficients in the “low” states are explicitly set April 7, 2009 DRAFT5 to zero (sparseness is explicitly imposed). The proposed Bayesian approach yields “error bars” on the rec... |

44 | Spike and Slab Variable Selection: Frequentist and Bayesian Strategies. The Annals of Statistics 33
- Ishwaran, Rao
- 2005
(Show Context)
Citation Context ...elet coefficients; both the high and low states are characterized by Gaussian statistics. In the model proposed here a different but related construction is employed, in terms of a “spike-slab” prior =-=[32]-=-, [33], [34], [35], [36], and here the coefficients in the “low” states are explicitly set April 7, 2009 DRAFT5 to zero (sparseness is explicitly imposed). The proposed Bayesian approach yields “erro... |

41 | Image denoising using Gaussian scale mixtures in the wavelet domain - Portilla, Strela, et al. - 2002 |

40 | Robust recovery of signals from a union of subspaces
- Eldar, Mishali
- 2008
(Show Context)
Citation Context ...r constrain or regularize the inversion, ideally reducing the number of required CS measurements N. This concept has been made rigorous recently for sparse θ [18], as well as for compressible θ [19], =-=[20]-=-, [21]; these papers demonstrate that one may achieve accurate CS inversions with substantially fewer projection measurements (smaller N) if known properties of the structure of April 7, 2009 DRAFT4 ... |

34 | Signal reconstruction using sparse tree representation
- La, Do
- 2005
(Show Context)
Citation Context ...lso employed in image denoising [22], as well as texture synthesis and image inpainting [23]. More directly related to CS, the wavelet tree structure has been employed in non-statistical CS inversion =-=[24]-=-, and within more statistical settings via the hidden Markov tree (HMT) [25]. There have also been methods that augment the CS sensing structure, with linkage to the scales in a wavelet decomposition ... |

31 | Compressed sensing and Bayesian experimental design
- Seeger, Nickisch
- 2008
(Show Context)
Citation Context ...ure statistically. The proposed technique is most closely related to recent Bayesian CS approaches for imposing prior belief about the signal of interest, usually in terms of a sparseness prior [15], =-=[30]-=- (this is closely related to more general research on imposing sparseness in Bayesian priors [31]). None of these previous Bayesian approaches explicitly imposed the known statistical structure of the... |

30 |
Single disperser design for coded aperture snapshot spectral imaging
- Wagadarikar, John, et al.
- 2008
(Show Context)
Citation Context ...rror proportional to the energy in the negligible coefficients [9]. Two of the early important applications of CS are in MRI [10] and in development of new hyperspectral cameras April 7, 2009 DRAFT3 =-=[11]-=-. Details on how to design the compressive-sensing projection vectors, and requirements on the (typically relatively small) number of such projections, may be found in [6], [7], [8], [9]. Assume that ... |

29 | Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization
- Duarte-Carvajalino, Sapiro
(Show Context)
Citation Context ... to account for prior knowledge about the properties of images. Researchers have also developed techniques that impose prior structure through learning the appropriate basis for sparse representation =-=[28]-=-, [29]. Therefore prior knowledge about images and the CS sensing process has been used previously, with this prior knowledge not limited to wavelets. While the above references impose various forms o... |

25 | Wavelet-domain compressive signal reconstruction using a hidden Markov tree model
- Duarte, Wakin, et al.
- 2008
(Show Context)
Citation Context ... inpainting [23]. More directly related to CS, the wavelet tree structure has been employed in non-statistical CS inversion [24], and within more statistical settings via the hidden Markov tree (HMT) =-=[25]-=-. There have also been methods that augment the CS sensing structure, with linkage to the scales in a wavelet decomposition [26]. In addition to imposing and exploiting prior knowledge about structure... |

24 | Perspectives on sparse bayesian learning
- Wipf, Palmer, et al.
- 2003
(Show Context)
Citation Context ...s for imposing prior belief about the signal of interest, usually in terms of a sparseness prior [15], [30] (this is closely related to more general research on imposing sparseness in Bayesian priors =-=[31]-=-). None of these previous Bayesian approaches explicitly imposed the known statistical structure of the wavelet decomposition of images, this constituting an important contribution of this paper. Whil... |

22 | Functional Clustering by Bayesian Wavelet Methods
- Ray, Mallick
- 2006
(Show Context)
Citation Context ...ts. This is a two-component mixture model, and the two components are associated with the two states in the HMT. Related models of this type have been employed previously for wavelet-based clustering =-=[39]-=-. The form of this model is different from an HMT [5] in that the coefficient associated with the “low” state is now explicitly set to zero, such that the inferred wavelet coefficients are explicitly ... |

14 | Sampling theorems for signals from the union of linear subspaces
- Blumensath, Davies
- 2007
(Show Context)
Citation Context ...cture in θ that may be exploited to further constrain or regularize the inversion, ideally reducing the number of required CS measurements N. This concept has been made rigorous recently for sparse θ =-=[18]-=-, as well as for compressible θ [19], [20], [21]; these papers demonstrate that one may achieve accurate CS inversions with substantially fewer projection measurements (smaller N) if known properties ... |

5 | Multi-task compressive sensing with Dirichlet process priors
- Qi, Liu, et al.
- 2008
(Show Context)
Citation Context ...een these different measurements, the total number of CS measurements may be reduced). The Bayesian setting proposed here is particularly amenable to the joint processing of data from multiple images =-=[37]-=-, and this will be investigated in future research. It is also of interest to examine the statistical October 16, 2008 DRAFT29 leveraging of structure in other popular transforms, such as the DCT. RE... |

4 |
JPEG2000 tutorial
- Christopoulos
- 1999
(Show Context)
Citation Context ...screte-cosine and wavelet transforms [1] constituting two important examples. The discrete cosine transform (DCT) is employed in the JPEG standard [2], with wavelets employed in the JPEG2000 standard =-=[3]-=-. Wavelet-based transform coding [4] explicitly exploits the structure [5] manifested in the wavelet coefficients of typical data. Specifically, for most natural data (signals and images) the wavelet ... |

4 |
basis compressed sensing
- Best
(Show Context)
Citation Context ...count for prior knowledge about the properties of images. Researchers have also developed techniques that impose prior structure through learning the appropriate basis for sparse representation [28], =-=[29]-=-. Therefore prior knowledge about images and the CS sensing process has been used previously, with this prior knowledge not limited to wavelets. While the above references impose various forms of prio... |

1 |
Annihilating flter-based decoding in the compressed sensing framework
- Hormati, Vetterli
- 2007
(Show Context)
Citation Context ...further constrain or regularize the inversion, ideally reducing the number of required CS measurements N. This concept has been made rigorous recently for sparse θ [18], as well as for compressible θ =-=[19]-=-, [20], [21]; these papers demonstrate that one may achieve accurate CS inversions with substantially fewer projection measurements (smaller N) if known properties of the structure of April 7, 2009 DR... |