## Parameter estimation in TV image restoration using variational distribution approximation (2008)

### Cached

### Download Links

- [decsai.ugr.es]
- [ivpl.ece.northwestern.edu]
- [ivpl.eecs.northwestern.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | IEEE TRANS. IMAGE PROCESSING |

Citations: | 45 - 27 self |

### BibTeX

@ARTICLE{Babacan08parameterestimation,

author = {S. Derin Babacan and Rafael Molina and Aggelos K. Katsaggelos},

title = {Parameter estimation in TV image restoration using variational distribution approximation},

journal = {IEEE TRANS. IMAGE PROCESSING},

year = {2008},

volume = {17},

number = {3},

pages = {326--339}

}

### Years of Citing Articles

### OpenURL

### Abstract

In this paper, we propose novel algorithms for total variation (TV) based image restoration and parameter estimation utilizing variational distribution approximations. Within the hierarchical Bayesian formulation, the reconstructed image and the unknown hyperparameters for the image prior and the noise are simultaneously estimated. The proposed algorithms provide approximations to the posterior distributions of the latent variables using variational methods. We show that some of the current approaches to TV-based image restoration are special cases of our framework. Experimental results show that the proposed approaches provide competitive performance without any assumptions about unknown hyperparameters and clearly outperform existing methods when additional information is included.

### Citations

1894 |
Numerical Optimization
- Nocedal, SJ
- 2000
(Show Context)
Citation Context ...s obtained, that is Using this approximation, the last two terms in (A11) can be expressed as (A10) Alternatively, a CG method can be applied. In our experiments we used the basic CG version shown in =-=[42]-=- to solve (A1). Note that several methods can be used (see, for instance, [38]–[40]) to calculate the TV image estimate without the use of the majorization of the TV prior. APPENDIX II CALCULATION OF ... |

1374 |
Nonlinear total variation based noise removal algorithms
- Rudin, Osher, et al.
- 1992
(Show Context)
Citation Context ...set of connected pixels) for the MRF, and is a potential function defined on a clique. A critical issue is the choice of the energy function. In this paper we use the total variation (TV) image prior =-=[6]-=- whose energy function is the discrete version of the total variation integral defined as We will explicitly write the form of the prior model in the next section. If the hyperparameters and are known... |

1240 |
Statistical decision theory and Bayesian analysis. Springer series in Statistics
- Berger
- 1985
(Show Context)
Citation Context ...the Bayesian literature is devoted to finding hyperprior distributions for which can be either calculated in a straightforward way or be closely approximated. These are the so called conjugate priors =-=[28]-=- which have Fig. 1. Graphical model showing relationships between variables. the intuitive feature of allowing one to begin with a certain functional form for the prior and end up with a posterior of ... |

1154 | Information theory and statistics
- Kullback
(Show Context)
Citation Context ...ng interest in the application of variational methods [19], [23] to inference problems. These methods attempt to approximate posterior distributions with the use of the Kullback–Leibler cross-entropy =-=[24]-=-. Application of variational methods to Bayesian inference problems include graphical models and neural networks [23], independent component analysis [19], mixtures of factor analyzers, linear dynamic... |

922 | On the statistical analysis of dirty pictures
- Besag
- 1986
(Show Context)
Citation Context ...m is equivalent to maximizing alternatively in the hyperparameters and image the lower bound of given in (24). In other words, the estimation procedure is an iterated conditional mode (ICM) algorithm =-=[36]-=-. To end this section, we comment on two particular hyperparameter distributions . The first one is obtained when both and are known quantities. Then Algorithm 2 with , , and , provides the same solut... |

831 | An introduction to variational methods for graphical models
- Jordan, Ghahramani, et al.
- 1999
(Show Context)
Citation Context ... and sampling methods [18]. The basic underlying idea, as will be explained later, is to approximate with a simpler distribution. See the very interesting [19], [20] books [21], [22] and book chapter =-=[23]-=- for a comprehensive introduction to variational methods. The last few years have seen a growing interest in the application of variational methods [19], [23] to inference problems. These methods atte... |

299 |
Spatial statistics
- Ripley
- 1981
(Show Context)
Citation Context ...CG) to find the BFO1 and BFO2 image estimates. We also included results obtained with the use of the algorithm in [16] which models the image distribution by a simultaneous autoregression (SAR) model =-=[37]-=- instead of a TV model and simultaneously estimates the prior and image hyperparameters. This algorithm will be denoted by MOL in the results. Comparing TV-based algorithms with this method provided u... |

275 | Variational algorithms for approximate bayesian inference
- Beal
- 2003
(Show Context)
Citation Context ...tion (see, for instance, [14] and [17]) and sampling methods [18]. The basic underlying idea, as will be explained later, is to approximate with a simpler distribution. See the very interesting [19], =-=[20]-=- books [21], [22] and book chapter [23] for a comprehensive introduction to variational methods. The last few years have seen a growing interest in the application of variational methods [19], [23] to... |

223 | An introduction to MCMC for machine learning
- Andrieu, Freitas, et al.
(Show Context)
Citation Context ...oach is provided by the variational distribution approximation. This approximation can be thought of as being between the Laplace approximation (see, for instance, [14] and [17]) and sampling methods =-=[18]-=-. The basic underlying idea, as will be explained later, is to approximate with a simpler distribution. See the very interesting [19], [20] books [21], [22] and book chapter [23] for a comprehensive i... |

215 | The Relevance Vector Machine
- Tipping
- 2000
(Show Context)
Citation Context ...oblems include graphical models and neural networks [23], independent component analysis [19], mixtures of factor analyzers, linear dynamic systems, hidden Markov models [20], support vector machines =-=[25]-=- and blind deconvolution problems (see [15], [26], and [27]). (7) (8) (9)328 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 3, MARCH 2008 In this paper, we use a TV prior distribution for the im... |

178 |
K.: “Digital Image Restoration
- Banham, Katsaggelos, et al.
- 1997
(Show Context)
Citation Context ...tems or extraterrestrial observations of the earth and the planets), commercial photography, medical imaging (e.g., X-rays, digital angiograms, autoradiographs), and molecular and cellular bioimaging =-=[2]-=-–[4]. The degradation can be due to the atmospheric turbulence, the relative motion between the camera and the scene, and the finite resolution of the acquisition instrument. A standard formulation of... |

105 | Bayesian parameter estimation via variational methods, Statistics and Computing 10
- Jaakkola, Jordan
- 1999
(Show Context)
Citation Context ...ion with is a very natural extension of the majorization-minimization approach to function optimization (see [29]) and that local majorization has also been applied to variational logistic regression =-=[30]-=-, as well as, to the inference of its parameters (see [31] and [32]). The following algorithm can, therefore, be used for calculating the approximating posteriors . Algorithm 1 Posterior parameter and... |

95 |
Image Processing And Analysis
- Chan, Shen
- 2005
(Show Context)
Citation Context ...P.2007.916051 (1) estimate of given , , and knowledge about and possibly [2]. A number of approaches have been developed in providing solutions to the restoration problem (see, for example, [2], [3], =-=[5]-=-, and references therein). A straightforward approach to the restoration problem is to use least squares estimation and select , an estimate of the original image, as where . However, as is well known... |

65 | Bayesian and regularization methods for hyperparameters estimate in image restoration - Molina, Katsaggelos, et al. - 1999 |

58 | robust total variation-based reconstruction of noisy, blurred images - Vogel, Oman, et al. - 1998 |

49 | Ensemble learning for independent component analysis
- Miskin
- 2000
(Show Context)
Citation Context ...roximation (see, for instance, [14] and [17]) and sampling methods [18]. The basic underlying idea, as will be explained later, is to approximate with a simpler distribution. See the very interesting =-=[19]-=-, [20] books [21], [22] and book chapter [23] for a comprehensive introduction to variational methods. The last few years have seen a growing interest in the application of variational methods [19], [... |

46 | Recent developments in total variation image restoration
- Chan, Esedoglu, et al.
- 2006
(Show Context)
Citation Context ...y to select, as the restoration of , the image defined by Not much work has been reported in the literature on the joint parameter and image estimation when the parameters and are not known (see [5], =-=[8]-=- for recent developments in variational modeling and inference). Rudin et al. [6] consider the minimization of constrained by , where represents an estimate of the noise variance, (2) (3) (4) (5) 1057... |

41 | Ensemble learning for blind image separation and deconvolution
- Miskin, MacKay
(Show Context)
Citation Context ...ks [23], independent component analysis [19], mixtures of factor analyzers, linear dynamic systems, hidden Markov models [20], support vector machines [25] and blind deconvolution problems (see [15], =-=[26]-=-, and [27]). (7) (8) (9)328 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 3, MARCH 2008 In this paper, we use a TV prior distribution for the image, and gamma distributions for the unknown para... |

36 | A Variational Approach for Bayesian Blind Image Deconvolution
- Likas, Galatsanos
- 2004
(Show Context)
Citation Context ...ndependent component analysis [19], mixtures of factor analyzers, linear dynamic systems, hidden Markov models [20], support vector machines [25] and blind deconvolution problems (see [15], [26], and =-=[27]-=-). (7) (8) (9)328 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 3, MARCH 2008 In this paper, we use a TV prior distribution for the image, and gamma distributions for the unknown parameter (hyp... |

35 |
A.: “Image restoration based on a subjective criterion
- Anderson, Netravali
(Show Context)
Citation Context ... probability distribution: 2) Find (27) (24) (28)330 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 3, MARCH 2008 3) Find Set (29) (30) matrix has also been referred to as the visibility matrix =-=[33]-=- since it describes the masking property of the human visual system, according to which noise is not visible in high spatial activity regions (its high frequencies are masked by the edges), while it i... |

35 | Cosine Transform Based Preconditioners for Total Variation Minimization
- Chan, Chan, et al.
- 1995
(Show Context)
Citation Context ...ms required on the average about 2–5 min on a 3.20-GHz Xeon PC for 256 256 images. Note that the running time of the algorithms can be improved by utilizing preconditioning methods (see, for example, =-=[38]-=-–[41]). VI. CONCLUSION We have presented two new methods for the simultaneous estimation of the image and the unknown hyperparameters in TV-based image restoration problems. We adopt a variational app... |

32 |
On the Hierarchical Bayesian Approach to Image Restoration: Applications to Astronomical Images
- Molina
- 1994
(Show Context)
Citation Context ...sis [12]) consists of estimating the hyperparameters , by using and then estimating the image by solving Another approach, also commonly used in image restoration, is the so called empirical analysis =-=[16]-=-, which consists of calculating the restoration by solving These inference procedures aim at optimizing a given function and not at obtaining posterior distributions that can be analyzed or simulated ... |

27 |
N.: A general framework for frequency domain multichannel signal processing
- Katsaggelos, Lay, et al.
- 1993
(Show Context)
Citation Context ...s convex combinations of their initial values and their maximum likelihood (ML) estimates. These ML estimates have been derived before either empirically or by using regularization formulations [34], =-=[35]-=-. According to (44) and (45), when they are equal to zero, no confidence is placed on the initial values of the hyperparameters and ML estimates are used, while when they are asymptotically equal to o... |

26 |
Deconvolution methods for 3-D fluorescence microscopy images
- Sarder, Nehorai
- 2006
(Show Context)
Citation Context ... or extraterrestrial observations of the earth and the planets), commercial photography, medical imaging (e.g., X-rays, digital angiograms, autoradiographs), and molecular and cellular bioimaging [2]–=-=[4]-=-. The degradation can be due to the atmospheric turbulence, the relative motion between the camera and the scene, and the finite resolution of the acquisition instrument. A standard formulation of the... |

24 |
Adaptive total-variation image deconvolution: A majorization-minimization approach,” presented at the EUSIPCO
- Bioucas-Dias, Figueiredo, et al.
- 2006
(Show Context)
Citation Context ...eed to estimate both the image and the associated Lagrange multiplier to this constrained optimization problem. Bertalmio et al. [9] make the Lagrange multiplier region dependent. Bioucas-Dias et al. =-=[10]-=-, using their majorization-minimization approach [11], propose a Bayesian method to estimate the original image and assuming that an estimate of the noise variance is available. To our knowledge no wo... |

21 | Blind deconvolution using a variational approach to parameter, image, and blur estimation
- Molina, Mateos, et al.
- 2006
(Show Context)
Citation Context ...tribution of the observation, , the unknown image, , and the hyperparameters and . To model the joint distribution, we utilize in this paper the hierarchical Bayesian paradigm (see, for example, [12]–=-=[15]-=-). In the hierarchical approach to image restoration, we have at least two stages. In the first stage, knowledge about the structural form of the observation noise and the structural behavior of the i... |

14 |
Bayesian Hierarchical Mixtures of Experts
- Bishop, Svensén
- 2003
(Show Context)
Citation Context ...inimization approach to function optimization (see [29]) and that local majorization has also been applied to variational logistic regression [30], as well as, to the inference of its parameters (see =-=[31]-=- and [32]). The following algorithm can, therefore, be used for calculating the approximating posteriors . Algorithm 1 Posterior parameter and image distributions estimation in TV restoration using . ... |

13 |
TV based image restoration with local constraints
- Bertalmio, Caselles, et al.
(Show Context)
Citation Context ...TORATION USING VARIATIONAL DISTRIBUTION APPROXIMATION 327 and then proceed to estimate both the image and the associated Lagrange multiplier to this constrained optimization problem. Bertalmio et al. =-=[9]-=- make the Lagrange multiplier region dependent. Bioucas-Dias et al. [10], using their majorization-minimization approach [11], propose a Bayesian method to estimate the original image and assuming tha... |

13 | Hyperparameter estimation in image restoration problems with partially known blurs
- Galatsanos, Mesarovic, et al.
- 2002
(Show Context)
Citation Context ... general a challenging task. An approach is provided by the variational distribution approximation. This approximation can be thought of as being between the Laplace approximation (see, for instance, =-=[14]-=- and [17]) and sampling methods [18]. The basic underlying idea, as will be explained later, is to approximate with a simpler distribution. See the very interesting [19], [20] books [21], [22] and boo... |

12 |
Spatially adaptive iterative algorithm for the restoration of astronomical images
- Katsaggelos, Kang
- 1995
(Show Context)
Citation Context ...quencies are masked by the edges), while it is visible in the low spatial frequency (flat) regions. The visibility matrix and its complementary matrix have been used in iterative image restoration in =-=[34]-=-. By differentiating the integral on the right hand side of (29) with respect to and setting it equal to zero, we obtain (37) Let us now further develop each of the steps of the above algorithm. To ca... |

11 |
The Variational Bayes Method
- Smídl, Quinn
- 2005
(Show Context)
Citation Context ...for instance, [14] and [17]) and sampling methods [18]. The basic underlying idea, as will be explained later, is to approximate with a simpler distribution. See the very interesting [19], [20] books =-=[21]-=-, [22] and book chapter [23] for a comprehensive introduction to variational methods. The last few years have seen a growing interest in the application of variational methods [19], [23] to inference ... |

10 | Unifying probabilistic and variational estimation
- Hamza, Krim, et al.
- 2002
(Show Context)
Citation Context ...version of the total variation integral defined as We will explicitly write the form of the prior model in the next section. If the hyperparameters and are known, following the Bayesian paradigm (see =-=[7]-=- for the unification of probabilistic and variational estimation), it is customary to select, as the restoration of , the image defined by Not much work has been reported in the literature on the join... |

8 |
A (2007) Superresolution image reconstruction using fast inpainting algorithms
- Chan, Ng, et al.
(Show Context)
Citation Context ...pressed as (A10) Alternatively, a CG method can be applied. In our experiments we used the basic CG version shown in [42] to solve (A1). Note that several methods can be used (see, for instance, [38]–=-=[40]-=-) to calculate the TV image estimate without the use of the majorization of the TV prior. APPENDIX II CALCULATION OF REQUIRED EXPECTED VALUES IN ALGORITHM 1 In this section we show how the calculation... |

7 |
Hierarchical Bayesian image restoration for partially-known blur
- Galatsanos, Mesarovic, et al.
- 2000
(Show Context)
Citation Context ...a challenging task. An approach is provided by the variational distribution approximation. This approximation can be thought of as being between the Laplace approximation (see, for instance, [14] and =-=[17]-=-) and sampling methods [18]. The basic underlying idea, as will be explained later, is to approximate with a simpler distribution. See the very interesting [19], [20] books [21], [22] and book chapter... |

7 |
Optimization,” in Springer Texts in Statistics
- Lange
- 2004
(Show Context)
Citation Context ... the process to find the best posterior distribution approximation of the image in combination with is a very natural extension of the majorization-minimization approach to function optimization (see =-=[29]-=-) and that local majorization has also been applied to variational logistic regression [30], as well as, to the inference of its parameters (see [31] and [32]). The following algorithm can, therefore,... |

6 | A Bayesian approach to estimate and transmit regularization parameters for reducing blocking artifacts - Mateos, Katsaggelos, et al. - 2000 |

5 | Total variation image restoration and parameter estimation using variational distribution approximation
- Babacan, Molina, et al.
- 2007
(Show Context)
Citation Context ...l de Ciencia y Tecnología” under contract TIC2007-65533 and in part by the Spanish research programme Consolider Ingenio 2010: MIPRCV (CSD2007-00018). Preliminary results of this work can be found in =-=[1]-=-. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Michael Elad. S. D. Babacan and A. K. Katsaggelos are with the Department of Electrical Engin... |

5 |
A total variation regularization based superresolution reconstruction algorithm for digital video
- Ng, Shen, et al.
- 2007
(Show Context)
Citation Context ...quired on the average about 2–5 min on a 3.20-GHz Xeon PC for 256 256 images. Note that the running time of the algorithms can be improved by utilizing preconditioning methods (see, for example, [38]–=-=[41]-=-). VI. CONCLUSION We have presented two new methods for the simultaneous estimation of the image and the unknown hyperparameters in TV-based image restoration problems. We adopt a variational approach... |