## Wavelet thresholding via mdl for natural images (2000)

### Cached

### Download Links

- [www.stat.ucla.edu]
- [oz.berkeley.edu]
- [www.stat.ucla.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | IEEE Transactions on Information Theory |

Citations: | 25 - 1 self |

### BibTeX

@ARTICLE{Hansen00waveletthresholding,

author = {Mark Hansen and Bin Yu},

title = {Wavelet thresholding via mdl for natural images},

journal = {IEEE Transactions on Information Theory},

year = {2000},

volume = {46},

pages = {1778--1788}

}

### Years of Citing Articles

### OpenURL

### Abstract

We study the application of Rissanen's Principle of Minimum Description Length (MDL) to the problem of wavelet denoising and compression for natural images. After making a con-nection between thresholding and model selection, we derive an MDL criterion based on a Laplacian model for noiseless wavelet coe cients. We nd that this approach leads to an adap-tive thresholding rule. While achieving mean squared error performance comparable with other popular thresholding schemes, the MDL procedure tends to keep far fewer coe cients. From this property, we demonstrate that our method is an excellent tool for simultaneous denoising and compression. We make this claim precise by analyzing MDL thresholding in two optimality frameworks; one in which we measure rate and distortion based on quantized coe cients and one in which we do not quantize, but instead record rate simply as the number of non-zero coe cients.

### Citations

2385 | A theory for multi-resolution signal decomposition: The wavelet representation
- Mallat
- 1989
(Show Context)
Citation Context ...ages. Extensive empirical work has led to the characterization that the wavelet coe cients derived from noiseless natural signals approximately follow a Laplacian or generalized Gaussian distribution =-=[19, 20, 29, 30]-=-. For this class of signals, it is well known that the universal threshold p 2 log n used by VisuShrink eliminates too many coe cients, while a variant of the SureShrink procedure, known as Sure, work... |

1728 |
Lectures on Wavelets
- Daubechies, Ten
- 1992
(Show Context)
Citation Context ... the number of pixels in a particular subband (usually a power of 2). All experimental results in this paper are based on a 3-level orthonormal wavelet decomposition using Daubechies' 8-symmelet (cf. =-=[10, 20, 35]-=-), and thresholding is applied only to the \detail" subbands HL, LH and HH at each level. Coe cients in LL(3) are estimated by their noisy counterparts. For simplicity, we assume that 2 is known and e... |

1251 |
Embedded image coding using zerotrees of wavelet coefficients
- Shapiro
- 1993
(Show Context)
Citation Context ...del. Finally, the rules proposed in this paper are based on an assumption that coe cients are iid from some population. Improvements are expected if one takes into account the spatial dependence (cf. =-=[31, 18, 4, 9, 37, 21]-=-). This can be done in our MDL framework by coding the model index using a Markov Random Field (MRF) instead of a Bernoulli coder. Especially appealing are the MRFs such as Chien's model which give ri... |

813 | De-noising by soft thresholding
- Donoho
- 1995
(Show Context)
Citation Context ...n, MDL, natural images, statistical estimation, wavelet thresholding. 1 Bell Laboratories, Murray Hill, NJ and 2 University of California at Berkeley, Berkeley, CAsI Introduction Donoho and Johnstone =-=[13, 12, 14]-=- proposed the use of wavelet thresholding for denoising 1-dimensional curves observed with additive, white noise. Their schemes are shown to be (essentially) minimax optimal in terms of mean squared e... |

682 | 1995], ‘Adapting to unknown smoothness via wavelet shrinkage - Donoho, Johnstone |

553 |
Ideal spatial adaptation via wavelet shrinkage. Biometrika 81
- Donoho, Johnstone
- 1994
(Show Context)
Citation Context ...n, MDL, natural images, statistical estimation, wavelet thresholding. 1 Bell Laboratories, Murray Hill, NJ and 2 University of California at Berkeley, Berkeley, CAsI Introduction Donoho and Johnstone =-=[13, 12, 14]-=- proposed the use of wavelet thresholding for denoising 1-dimensional curves observed with additive, white noise. Their schemes are shown to be (essentially) minimax optimal in terms of mean squared e... |

499 | Stochastic Complexity - RISSANEN - 1989 |

474 |
Wavelets and subband coding
- Vetterli, Kovačevic
- 1995
(Show Context)
Citation Context ...ails and proofs of the results in Section III are collected in appendices at the end of the paper. Throughout our presentation, we will use standard terminology from wavelet image subband coding (cf. =-=[35]-=-). II Denoising via Thresholding and Model Selection Wavelet thresholding for signal denoising sets \small" coe cients to zero, yielding an analytical compression of the signal (cf. [13, 12, 14, 11, 5... |

325 | Wavelet-based statistical signal processing using hidden Markov models
- Crouse, Nowak, et al.
- 1998
(Show Context)
Citation Context ...4]), and the MSE-optimal threshold derived in Corollary 1; although other less restrictive (and more expensive in terms of coe cient count) denoising techniques might give better MSE performance (cf. =-=[9]-=-). The MSE's in this table are quite comparable for all four methods. The threshold Tbayes and Topt perform the best in terms of MSE by at most 10% over Tmap and lMDL, while lMDL requires 12sBaboon Tm... |

306 | The minimum description length principle in coding and modeling,” Information Theory
- Barron, Rissanen, et al.
- 1998
(Show Context)
Citation Context ...Shrink. B Model Classes and Analytical Compression As a principle, MDL suggests that we select a model or model class that yields the shortest description of a dataset. Two recent review articles are =-=[2, 17]-=-: the rst geared toward an audience versed in information theory and the second written for the statistics community. The MDL philosophy is descriptive in the sense that models are viewed as a means o... |

294 |
Estimation of the mean of a multivariate normal distribution
- Stein
- 1981
(Show Context)
Citation Context ...ink, was also proposed and shown to be minimax optimal over signals with unknown smoothness (belonging to one of a range of Besov spaces). To be more speci c, recall Stein's unbiased estimate of risk =-=[33]-=- n , 2 nX i=1 I fi:jyij tg + nX i=1 (jyij^t) 2 : (4) Letting T denote the value that minimizes (4), Donoho and Johnstone [14] de ne the SureShrink soft-threshold T to be T = min(T; p 2 log n): (5) The... |

236 | Adaptive Wavelet Thresholding for Image Denoising and Compression
- Chang, Yu, et al.
(Show Context)
Citation Context ...l describe Sure in Section II). Bayesian approaches that make use of the distributional characterization for natural images have yielded soft-thresholding rules that match the performance of Sure (cf.=-=[3, 26, 24]-=-). This similarity should be expected given that these schemes and Sure both attempt to minimize the same Bayes risk [3]. In general, wavelet thresholding can be thought of as a special case of statis... |

203 | Noise removal via bayesian wavelet coring
- Simoncelli, Adelson
- 1996
(Show Context)
Citation Context ...ages. Extensive empirical work has led to the characterization that the wavelet coe cients derived from noiseless natural signals approximately follow a Laplacian or generalized Gaussian distribution =-=[19, 20, 29, 30]-=-. For this class of signals, it is well known that the universal threshold p 2 log n used by VisuShrink eliminates too many coe cients, while a variant of the SureShrink procedure, known as Sure, work... |

201 | Wavelet thresholding via a Bayesian approach
- Abramovich, Sapatinas, et al.
- 1998
(Show Context)
Citation Context ...re to be estimated. Since the pioneering work of Donoho and Johnstone, many variants and improvements of their thresholding rules have appeared in the literature on statistical curve estimation (e.g. =-=[6,1,7,8,28,36]-=-). In this article we consider so-called natural images. Extensive empirical work has led to the characterization that the wavelet coe cients derived from noiseless natural signals approximately follo... |

195 | Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage
- Chambolle, DeVore, et al.
- 1998
(Show Context)
Citation Context ...d coding (cf. [35]). II Denoising via Thresholding and Model Selection Wavelet thresholding for signal denoising sets \small" coe cients to zero, yielding an analytical compression of the signal (cf. =-=[13, 12, 14, 11, 5]-=-). In a series of papers, Donoho and Johnstone [13, 12, 14] assume the following additive, white noise model in the wavelet domain: yi = i + i; i =1;::: ;n (1) 2swhere the i are wavelet coe cients of ... |

181 | Spatially adaptive wavelet thresholding with context modeling for image denoising
- Chang, Yu, et al.
- 2000
(Show Context)
Citation Context ...del. Finally, the rules proposed in this paper are based on an assumption that coe cients are iid from some population. Improvements are expected if one takes into account the spatial dependence (cf. =-=[31, 18, 4, 9, 37, 21]-=-). This can be done in our MDL framework by coding the model index using a Markov Random Field (MRF) instead of a Bernoulli coder. Especially appealing are the MRFs such as Chien's model which give ri... |

176 | Analysis of multiresolution image denoising schemes using generalized Gaussian and complexity priors
- Moulin, Liu
- 1999
(Show Context)
Citation Context ...l describe Sure in Section II). Bayesian approaches that make use of the distributional characterization for natural images have yielded soft-thresholding rules that match the performance of Sure (cf.=-=[3, 26, 24]-=-). This similarity should be expected given that these schemes and Sure both attempt to minimize the same Bayes risk [3]. In general, wavelet thresholding can be thought of as a special case of statis... |

173 |
Adaptive Bayesian wavelet shrinkage
- Chipman, Kolaczyk, et al.
- 1997
(Show Context)
Citation Context ...re to be estimated. Since the pioneering work of Donoho and Johnstone, many variants and improvements of their thresholding rules have appeared in the literature on statistical curve estimation (e.g. =-=[6,1,7,8,28,36]-=-). In this article we consider so-called natural images. Extensive empirical work has led to the characterization that the wavelet coe cients derived from noiseless natural signals approximately follo... |

152 | Image coding based on mixture modeling of wavelet coefficients and a fast estimation-quantization framework
- LoPresto, Ramchandran, et al.
- 1997
(Show Context)
Citation Context ...del. Finally, the rules proposed in this paper are based on an assumption that coe cients are iid from some population. Improvements are expected if one takes into account the spatial dependence (cf. =-=[31, 18, 4, 9, 37, 21]-=-). This can be done in our MDL framework by coding the model index using a Markov Random Field (MRF) instead of a Bernoulli coder. Especially appealing are the MRFs such as Chien's model which give ri... |

151 | Lowcomplexity image denoising based on statistical modeling of wavelet co152
- Mihcak, Kozintsev, et al.
- 1999
(Show Context)
Citation Context ...een in the R-E and R-D analyses. In contrast, when these models have been applied locally, to denoise a sample of pixels in a small neighborhood, there has been no observed improvement in performance =-=[21]-=-. Given the small sample sizes used in the local model of [21], it is di cult to discriminate between a logspline and Laplacian model. Finally, the rules proposed in this paper are based on an assumpt... |

142 | Polynomial splines and their tensor products in extended linear modeling,” The Annals of Statistics
- Stone, Hansen, et al.
- 1997
(Show Context)
Citation Context ...ed-form expressions for almost every quantity required in the denoising process. This suggests a two-piece Laplacian, also known in the statistics literature as a (linear) logspline distribution (see =-=[34]-=- for an overview and applications of this family). To generate Fig. 3, we t a linear logspline model with two symmetrically placed knots or breakpoints: Logspline : 2 e ,(a,b)jxj,b(jxj, )+ h e (b,a) 1... |

118 | Multiple shrinkage and subset selection in wavelets
- Clyde, Parmigiani, et al.
- 1998
(Show Context)
Citation Context ...re to be estimated. Since the pioneering work of Donoho and Johnstone, many variants and improvements of their thresholding rules have appeared in the literature on statistical curve estimation (e.g. =-=[6,1,7,8,28,36]-=-). In this article we consider so-called natural images. Extensive empirical work has led to the characterization that the wavelet coe cients derived from noiseless natural signals approximately follo... |

115 | Nonlinear wavelet shrinkage with Bayes rules and Bayes factors
- Vidakovic
- 1998
(Show Context)
Citation Context |

89 | Bayesian denoising of visual images in the wavelet domain
- Simoncelli
- 1999
(Show Context)
Citation Context ...ages. Extensive empirical work has led to the characterization that the wavelet coe cients derived from noiseless natural signals approximately follow a Laplacian or generalized Gaussian distribution =-=[19, 20, 29, 30]-=-. For this class of signals, it is well known that the universal threshold p 2 log n used by VisuShrink eliminates too many coe cients, while a variant of the SureShrink procedure, known as Sure, work... |

68 | Simultaneous noise suppression and signal compression using a library of orthonormal bases and the minimum description length criterion
- Saito
- 1994
(Show Context)
Citation Context ...hresholding can be thought of as a special case of statistical model selection where we have asmany (orthogonal) predictor variables (corresponding to wavelet basis elements) as there are data points =-=[22, 23, 32, 7, 8]-=-. Viewing the problem in this way is attractive because it allows one to separate the \kill" action (setting coe cients to zero) from the \keep" action (estimating the remaining coe cients via shrinka... |

42 | Fast wavelet techniques for near optimal image processing
- DeVore, Lucier
- 1992
(Show Context)
Citation Context ...d coding (cf. [35]). II Denoising via Thresholding and Model Selection Wavelet thresholding for signal denoising sets \small" coe cients to zero, yielding an analytical compression of the signal (cf. =-=[13, 12, 14, 11, 5]-=-). In a series of papers, Donoho and Johnstone [13, 12, 14] assume the following additive, white noise model in the wavelet domain: yi = i + i; i =1;::: ;n (1) 2swhere the i are wavelet coe cients of ... |

42 |
Filtering random noise from deterministic signals via data compression
- Natarajan
(Show Context)
Citation Context ... the percentage of non-zero coe cients as the analytical compression rate. This quantity is proportional to the actual coding rate if a xed quantizer is applied to the surviving coe cients. Natarajan =-=[25]-=- argues that a good compression algorithm should act as a denoiser, providing the distortion level matches the standard deviation of the noise. We follow Chang et al. [3] and consider quantizers havin... |

39 | Wavelet Thresholding Techniques for Power Spectrum Estimation
- Moulin
- 1994
(Show Context)
Citation Context ...hresholding can be thought of as a special case of statistical model selection where we have asmany (orthogonal) predictor variables (corresponding to wavelet basis elements) as there are data points =-=[22, 23, 32, 7, 8]-=-. Viewing the problem in this way is attractive because it allows one to separate the \kill" action (setting coe cients to zero) from the \keep" action (estimating the remaining coe cients via shrinka... |

34 | Empirical Bayes estimation in wavelet nonparametric regression. In
- Clyde, George
- 1999
(Show Context)
Citation Context |

27 | A Unified Approach for Encoding Clean and Noisy Sources by Means of Waveform and Autoregressive Model Vector Quantization - Ephraim, Gray - 1988 |

25 | A wavelet regularization method for diffuse radar-target imaging and speckle-noise reduction - Moulin - 1993 |

16 | A Bayesian Decision Theoretic Approach to Wavelet Thresholding
- Ruggeri, Vidakovic
- 1998
(Show Context)
Citation Context |

14 |
Model Selection and the Minimum Description Length Principle
- Hansen, Yu
- 2001
(Show Context)
Citation Context ...Shrink. B Model Classes and Analytical Compression As a principle, MDL suggests that we select a model or model class that yields the shortest description of a dataset. Two recent review articles are =-=[2, 17]-=-: the rst geared toward an audience versed in information theory and the second written for the statistics community. The MDL philosophy is descriptive in the sense that models are viewed as a means o... |

7 |
A wavelet regularization method for di#use radar-target and speckle-noise reduction
- Moulin
- 1993
(Show Context)
Citation Context ...hresholding can be thought of as a special case of statistical model selection where we have asmany (orthogonal) predictor variables (corresponding to wavelet basis elements) as there are data points =-=[22, 23, 32, 7, 8]-=-. Viewing the problem in this way is attractive because it allows one to separate the \kill" action (setting coe cients to zero) from the \keep" action (estimating the remaining coe cients via shrinka... |

5 |
Estimées locales fortement homogénes
- Nikolova
- 1997
(Show Context)
Citation Context ...l describe Sure in Section II). Bayesian approaches that make use of the distributional characterization for natural images have yielded soft-thresholding rules that match the performance of Sure (cf.=-=[3, 26, 24]-=-). This similarity should be expected given that these schemes and Sure both attempt to minimize the same Bayes risk [3]. In general, wavelet thresholding can be thought of as a special case of statis... |

5 |
Image subband coding using context based classi cation and adaptive quantization
- Yoo, Ortega, et al.
- 1999
(Show Context)
Citation Context ...y, the so-called Laplacian or double exponential: Lap : 2 e , jxj ; x 2 R : (7) In image compression studies, this simpli cation does not usually produce a noticeable degradation in performance (e.g. =-=[37]-=-), and so we also adopt a Laplacian model. We assume that the coe cients for a given subband are an iid sample from the distribution (7), where is to be determined for each subband separately. This re... |

2 |
A Wavelet Tour of Singal Processing
- Mallat
- 1998
(Show Context)
Citation Context |

1 |
A uni ed approach for encoding clean and noisy sources by means of waveform and autoregressive model vector quantization
- Ephraim, Gray
- 1988
(Show Context)
Citation Context ...stortion with respect to : Q = argmin QEd( ;Q(y)) (17) for some distortion measure d( ; ). The expectation in the expression above is with respect to the joint distribution of and y. Ephraim and Gray =-=[15]-=- derive a simple solution to this problem by rewriting the expectation in (17) as Ed( ;Q(y)) = Ed 0 (y; Q(y)) ; where d 0 (y; Q(y)) = E f d ( ;Q(y)) jyg is a new distortion measure comparing y and Q(y... |