## MDL Denoising (1999)

Venue: | IEEE Transactions on Information Theory |

Citations: | 53 - 10 self |

### BibTeX

@ARTICLE{Rissanen99mdldenoising,

author = {Jorma Rissanen and J. Rissanen},

title = {MDL Denoising},

journal = {IEEE Transactions on Information Theory},

year = {1999},

volume = {46},

pages = {2537--2543}

}

### Years of Citing Articles

### OpenURL

### Abstract

The so-called denoising problem, relative to normal models for noise, is formalized such that `noise' is defined as the incompressible part in the data while the compressible part defines the meaningful information bearing signal. Such a decomposition is effected by minimization of the ideal code length, called for by the Minimum Description Length (MDL) principle, and obtained by an application of the normalized maximum likelihood technique to the primary parameters, their range, and their number. For any orthonormal regression matrix, such as defined by wavelet transforms, the minimization can be done with a threshold for the squared coefficients resulting from the expansion of the data sequence in the basis vectors defined by the matrix. keywords: linear regression, wavelet transforms, threshold, stochastic complexity, Kolmogorov sufficient statistics 1 Introduction Intuitively speaking the so-called `denoising' problem is to separate an observed data sequence x 1 ; x 2 ; ...

### Citations

9133 |
Elements of Information Theory
- Cover, Thomas
- 1991
(Show Context)
Citation Context ...otic formula in [9] that is applicable to more general parametric model classes. The resulting decomposition is similar to Kolmogorov's sufficient statistics in the algorithmic theory of information, =-=[2]-=-,[1], [10], [11], and it will also be seen to extend the usual sufficient statistics decomposition of parametric likelihood functions of exponential type. Because the NML criterion involves the sum of... |

909 | Ideal spatial adaptation by wavelet shrinkage, Biometrika 81
- Donoho, Johnstone
- 1994
(Show Context)
Citation Context ...d data sequence x 1 ; x 2 ; : : : ; xn into a `meaningful' signalsx i and the remaining `noise' e i thus x i =sx i + e i . Taking the traditional approach Donoho and Johnstone in the pioneering paper =-=[4]-=- on application of wavelets to statistics posed this as the problem of estimating a function f(t i ) from its noisy values x i = f(t i ) + ffl i ; (1) where the noise ffl i is a normal, independent, i... |

533 |
Stochastic Complexity
- Rissanen
- 1989
(Show Context)
Citation Context ...itrary. In [7] a different approach to the denoising problem was studied based on an early version of the formula for the shortest code length for the observed sequence required in the MDL principle, =-=[10]-=-. In the notations above it is the same as that for the deviations x i \Gammasx i = e i , which is determined by the way they are modeled. Two types of distributions for them were considered, the firs... |

290 |
Fisher information and stochastic complexity
- Rissanen
- 1996
(Show Context)
Citation Context ...act formula exists even nonasymptotically. Such an NML criterion is derived by a two-fold extension of the normalization procedure done by Dom, [3], which, in turn, sharpens the asymptotic formula in =-=[9]-=- that is applicable to more general parametric model classes. The resulting decomposition is similar to Kolmogorov's sufficient statistics in the algorithmic theory of information, [2],[1], [10], [11]... |

156 | Model selection and the principle of minimum description length
- Hansen, Yu
- 2001
(Show Context)
Citation Context ...the two parameters R ands0 , which clearly affect the criterion (15) in an essential manner, or rather we replace them with other parameters which do not influence the relevant criterion. In [11] and =-=[6]-=- this was done simply by setting the two parameters to the values that minimize (15): R = R, ands0 =s, where R = n \Gamma1sfi 0 (x)\Sigma fi(x). However, the resulting f(x; fl;s(x); R(x)) is not a den... |

139 | The risk inflation criterion for multiple regression - Foster, George - 1994 |

84 | Wavelet shrinkage using cross-validation
- Nason
- 1996
(Show Context)
Citation Context ...(x \Gammasx) + ( x \Gamma x)] ! 2( +M ) no matter what the threshold is. 4 Examples We calculate two examples using wavelets defined by Daubechies' N=6 scaling function. The first example, taken from =-=[8]-=-, is a case where the MDL threshold is close to the ones in [4] as well as to the threshold in [8], obtained with a rather complicated cross-validation technique. It is clear from the criterion (40) t... |

60 | Hypothesis selection and testing by the MDL principle
- Rissanen
- 1999
(Show Context)
Citation Context ... [9] that is applicable to more general parametric model classes. The resulting decomposition is similar to Kolmogorov's sufficient statistics in the algorithmic theory of information, [2],[1], [10], =-=[11]-=-, and it will also be seen to extend the usual sufficient statistics decomposition of parametric likelihood functions of exponential type. Because the NML criterion involves the sum of the squares of ... |

33 |
Minimax description length for signal denoising and optimized representation
- Krim, Schick
- 1999
(Show Context)
Citation Context ...in turn, depends on the estimated variance of the noise. Clearly, there cannot be any unique way to imagine or model a signalsx i , which means that any estimate of its variance must be arbitrary. In =-=[7]-=- a different approach to the denoising problem was studied based on an early version of the formula for the shortest code length for the observed sequence required in the MDL principle, [10]. In the n... |

10 |
MDL estimation for small sample sizes and its application to linear regression
- Dom
- 1996
(Show Context)
Citation Context ...ximum Likelihood (NML) density function, for which an exact formula exists even nonasymptotically. Such an NML criterion is derived by a two-fold extension of the normalization procedure done by Dom, =-=[3]-=-, which, in turn, sharpens the asymptotic formula in [9] that is applicable to more general parametric model classes. The resulting decomposition is similar to Kolmogorov's sufficient statistics in th... |

9 |
Universal Sequential Coding of Single Messagesâ€™, Translated from Problems of
- unknown authors
- 1987
(Show Context)
Citation Context ...))dy ; (5) where y is restricted to the set Y ( 0 ; R) = fyj(y)s0 ; fi 0 (y)\Sigma fi(y)snRg: (6) In this the parameterss0 and R are such that the ML estimates fall within Y ( 0 ; R). It was shown in =-=[13]-=- that a normalization process like in (5) givessf (x; fl) as the solution to the minmax problem min q max x ln f(x; fl;sfi(x);s(x)) q(x) : The same density function was recently shown to solve even th... |

7 |
The MDL Principle in Modeling and Codingâ€™, special issue of
- Barron, Rissanen, et al.
- 1998
(Show Context)
Citation Context ... formula in [9] that is applicable to more general parametric model classes. The resulting decomposition is similar to Kolmogorov's sufficient statistics in the algorithmic theory of information, [2],=-=[1]-=-, [10], [11], and it will also be seen to extend the usual sufficient statistics decomposition of parametric likelihood functions of exponential type. Because the NML criterion involves the sum of the... |

1 |
A Generalized MinMax Bound for Universal Coding
- Rissanen
- 2000
(Show Context)
Citation Context ...x)) q(x) : The same density function was recently shown to solve even the following minmax problem min q max g E g log f(X n ;s`(X n ); fl) q(X n ) ; (7) 4 where q and g range over any distributions, =-=[12]-=-. Hence, in partiular g is not restricted to be of the normal type (3). In words:sf(x n ; fl) is the unique density function which minimizes over all density functions q(x n ) the mean ideal code leng... |