## Bayesian Separation of Images Modeled With MRFs Using MCMC

Citations: | 6 - 2 self |

### BibTeX

@MISC{Kayabol_bayesianseparation,

author = {Koray Kayabol and Student Member and Ercan E. Kuruo˘glu and Senior Member and Bülent Sankur and Senior Member},

title = {Bayesian Separation of Images Modeled With MRFs Using MCMC},

year = {}

}

### OpenURL

### Abstract

Abstract—We investigate the source separation problem of random fields within a Bayesian framework. The Bayesian formulation enables the incorporation of prior image models in the estimation of sources. Due to the intractability of the analytical solution, we resort to numerical methods for the joint maximization of the a posteriori distribution of the unknown variables and parameters. We construct the prior densities of pixels using Markov random fields based on a statistical model of the gradient image, and we use a fully Bayesian method with modified-Gibbs sampling. We contrast our work to approximate Bayesian solutions such as Iterated Conditional Modes (ICM) and to non-Bayesian solutions of ICA variety. The performance of the method is tested on synthetic mixtures of texture images and astrophysical images under various noise scenarios. The proposed method is shown to outperform significantly both its approximate Bayesian and non-Bayesian competitors. Index Terms—Astrophysical images, Bayesian source separation,

### Citations

1367 |
Idependent component analysis - a new concept
- Comon
- 1994
(Show Context)
Citation Context ... limited to linear schemes since for the applications in case, linear mixing forms a good first order approximation. A common methodology for source separation is Independent Component Analysis (ICA) =-=[10]-=-. In this study, we use the term “ICA-based methods” to represent the classical source separation methods and consider the Bayesian source separation to be a more general framework which includes the ... |

549 |
Markov Chain Monte Carlo in Practice
- Gilks, Richardson, et al.
- 1996
(Show Context)
Citation Context ...n in (9). However, direct sampling from the joint distribution, , is not possible. We, therefore, use Gibbs sampler to break down the multivariate sampling problem into a set of univariate ones [38], =-=[39]-=-. In other words, sampling from conditional distribution of all the unknowns is conducted separately from their corresponding marginal densities. When this iterative procedure converges, then the samp... |

436 | A fast fixed-point algorithm for independent component analysis - Hyvärinen, Oja - 1997 |

298 |
An information maximization approach to blind separation and blind deconvolution
- Bell, Sejnowski
- 1995
(Show Context)
Citation Context ...ng ICA-based methods for obtaining independent sources. A popular approach is to minimize the mutual information between the sources to be separated by adjusting the coefficients of the mixing matrix =-=[11]-=-. In another approach, the marginal likelihood function of the mixing matrix is maximized [12]. The third approach is based on the maximization of the non-Gaussianity of the sources [13]. The fourth a... |

240 | A generalized Gaussian image model for edge-preserving MAP estimation
- Bouman, Sauer
- 1993
(Show Context)
Citation Context ...rocessing in the absence of ground truth information. Several cases of inverse problems can be given. In image de-noising and restoration, the source image is estimated from its noisy observation [1]–=-=[3]-=-. In super-resolution image reconstruction and image fusion, the source image is estimated from multiple observations [4], [5]. In image separation multiple, source images are extracted from their mul... |

231 | Deterministic edge-preserving regularization in computed imaging
- Charbonnier, Blanc-Féraud, et al.
- 1997
(Show Context)
Citation Context ...ge processing in the absence of ground truth information. Several cases of inverse problems can be given. In image de-noising and restoration, the source image is estimated from its noisy observation =-=[1]-=-–[3]. In super-resolution image reconstruction and image fusion, the source image is estimated from multiple observations [4], [5]. In image separation multiple, source images are extracted from their... |

226 | An introduction to mcmc for machine learning
- Andrieu, Freitas, et al.
- 2003
(Show Context)
Citation Context ...ons. Notice that our scheme is not the classical Gibbs sampling algorithm because we embed Metropolis steps in it for sampling from source images. Consequently we denote it as modified Gibbs sampling =-=[41]-=- or Gibbs sampling with embedded Metropolis. The steps in Table II are iterated until convergence. It is beneficial to reduce the unknown space since with smaller number of unknowns the Gibbs sampling... |

222 | Independent factor analysis
- Attias
- 1999
(Show Context)
Citation Context ...ons of this strategy on different sources models list as follows. The parameters of mixture of Gaussians have been obtained by the EM method as in Moulines et al. [20], Snoussi et al. [21] and Attias =-=[22]-=-, and by variational EM by Miskin [23]. Miskin [23] and Kuruoglu et al. [24] have applied their approach to various image separation problems. For complex models, integration is not tractable. In the ... |

208 | Restoration of single super-resolution image from several blurred, noisy and down-sampled measured images
- Elad, Feuer
- 1997
(Show Context)
Citation Context ...restoration, the source image is estimated from its noisy observation [1]–[3]. In super-resolution image reconstruction and image fusion, the source image is estimated from multiple observations [4], =-=[5]-=-. In image separation multiple, source images are extracted from their multiple mixed observations. This is closely related to the Blind Source Separation (BSS) problem where the task is to reconstruc... |

204 |
Super-resolution image reconstruction: a technical overview
- Park, Kang
- 2003
(Show Context)
Citation Context ... and restoration, the source image is estimated from its noisy observation [1]–[3]. In super-resolution image reconstruction and image fusion, the source image is estimated from multiple observations =-=[4]-=-, [5]. In image separation multiple, source images are extracted from their multiple mixed observations. This is closely related to the Blind Source Separation (BSS) problem where the task is to recon... |

201 | A blind source separation technique using second-order statistics
- Belouchrani, Abed-Meraim, et al.
- 1997
(Show Context)
Citation Context .... The third approach is based on the maximization of the non-Gaussianity of the sources [13]. The fourth approach uses the time correlation or non whiteness of the sources to make separation possible =-=[15]-=-, [16]. The algorithms in this category drop the non-Gaussianity assumption of all sources and compensate for the missing information using the time correlation. The SOBI (Second Order Blind Identific... |

163 | Analysis of fMRI data by blind separation into independent spatial components
- McKeown, Makeig, et al.
- 1998
(Show Context)
Citation Context ...n this paper, we address the image source separation problem. Some current applications of image source separation include astrophysical images [6], document processing [7], and analysis of fMRI data =-=[8]-=-. The classical blind source separation techniques are based on the assumption of source independence and aim to recover unknown sources in the lack of information about the mixing operator. Generally... |

88 |
Numerical Bayesian Methods Applied to Signal Processing
- O’Ruaniaidh, Fitzgerald
- 1996
(Show Context)
Citation Context ...s given in (9). However, direct sampling from the joint distribution, , is not possible. We, therefore, use Gibbs sampler to break down the multivariate sampling problem into a set of univariate ones =-=[38]-=-, [39]. In other words, sampling from conditional distribution of all the unknowns is conducted separately from their corresponding marginal densities. When this iterative procedure converges, then th... |

85 | Nonlinear independent component analysis: existence and uniqueness results, Neural Networks 12 (3
- Hyvarinen, Pajunen
- 1999
(Show Context)
Citation Context ... of information about the mixing operator. Generally, in the observation model, the mixing operator is assumed to be linear while there are also works in the literature that consider nonlinear models =-=[9]-=-. Our work is limited to linear schemes since for the applications in case, linear mixing forms a good first order approximation. A common methodology for source separation is Independent Component An... |

49 | Ensemble learning for independent component analysis
- Miskin
- 2000
(Show Context)
Citation Context ...ces models list as follows. The parameters of mixture of Gaussians have been obtained by the EM method as in Moulines et al. [20], Snoussi et al. [21] and Attias [22], and by variational EM by Miskin =-=[23]-=-. Miskin [23] and Kuruoglu et al. [24] have applied their approach to various image separation problems. For complex models, integration is not tractable. In the second approach, one uses the joint po... |

49 | A comparison of algorithms for inference and learning in probabilistic graphical models
- Frey, Jojic
(Show Context)
Citation Context ...SITIVE RANDOM NUMBER IN THE UNIT INTERVAL; w: GENERATED SAMPLE TO BE TRIED; r: ACCEPTANCE RATIO OF THE GENERATED SAMPLE (9) (10) (11) The given procedure in (9)–(11) represents ICM update steps [19], =-=[37]-=-. If the direct solution of this maximization is not possible, iterative optimization methods can be used, such as steepest descent or Newton–Raphson. Difficulties may arise in MAP estimation due to n... |

47 | A bayesian approach to source separation
- Knuth
- 1999
(Show Context)
Citation Context ...ete sources. Mohammad-Djafari [18] derived the posterior in terms of the likelihood function and assigned prior probability density functions (pdf) to sources and the mixing matrix. Independently, in =-=[17]-=-, Knuth showed that the blind source separation problem can be formulated in a fully Bayesian framework. In [19], Rowe proposed a computational scheme for the solution of this problem formulated in th... |

31 | The three easy routes to independent component analysis: contrasts and geometry
- Cardoso
- 2001
(Show Context)
Citation Context ...third approach is based on the maximization of the non-Gaussianity of the sources [13]. The fourth approach uses the time correlation or non whiteness of the sources to make separation possible [15], =-=[16]-=-. The algorithms in this category drop the non-Gaussianity assumption of all sources and compensate for the missing information using the time correlation. The SOBI (Second Order Blind Identification)... |

31 | Fast joint separation and segmentation of mixed images
- Snoussi, Mohammad-Djafari
- 2004
(Show Context)
Citation Context ... are modeled with convex and nonconvex energy functions, they are estimated using a deterministic optimization technique, and then mixing matrix is updated using the Metropolis method. Snoussi et al. =-=[31]-=- proposed the Gaussian MRF model for source images and implemented a fast MCMC algorithm for joint separation and segmentation of mixed images. Kuruoglu et al. [32] gave a MRF formulation and used it ... |

28 | A maximum likelihood source separation for discrete sources
- Belouchrani, Cardoso
- 1994
(Show Context)
Citation Context ...mutual information between the sources to be separated by adjusting the coefficients of the mixing matrix [11]. In another approach, the marginal likelihood function of the mixing matrix is maximized =-=[12]-=-. The third approach is based on the maximization of the non-Gaussianity of the sources [13]. The fourth approach uses the time correlation or non whiteness of the sources to make separation possible ... |

27 |
On underdetermined source separation
- TALEB, JUTTEN
- 1999
(Show Context)
Citation Context ...osed by Moudden et al. [26]. The wavelet domain source separation has been also studied in [27] and [28]. Another frequently used iid model is mixture of Gaussian densities [20]–[24]. Hosseini et al. =-=[29]-=- proposed a method to separate sources based on a Markov chain model, so that the time correlation information of the sources could be used. Although some of these methods have been applied to 2-D ima... |

20 | A Bayesian approach to blind source separation
- Rowe
- 2002
(Show Context)
Citation Context ...probability density functions (pdf) to sources and the mixing matrix. Independently, in [17], Knuth showed that the blind source separation problem can be formulated in a fully Bayesian framework. In =-=[19]-=-, Rowe proposed a computational scheme for the solution of this problem formulated in the Bayesian framework utilizing ICM and Gibbs sampling procedures. The Bayesian BSS algorithms can be considered ... |

19 | Gaussian moments for noisy independent component analysis, IEEE signal processing letters, 2007. Mechanical Engineering 10.4028/www.scientific.net/AMM.26-28 Robust Method for Separation of Noisy Biomedical Signals
- Hyvarinen
(Show Context)
Citation Context ...mixing matrix [11]. In another approach, the marginal likelihood function of the mixing matrix is maximized [12]. The third approach is based on the maximization of the non-Gaussianity of the sources =-=[13]-=-. The fourth approach uses the time correlation or non whiteness of the sources to make separation possible [15], [16]. The algorithms in this category drop the non-Gaussianity assumption of all sourc... |

19 | A Bayesian approach to source separation
- Mohammad-Djafari
- 1999
(Show Context)
Citation Context ...optimization of all parameters. In [12], Belouchrani and Cardoso considered maximization of likelihood via Expectation-Maximization (EM) algorithm for separation of discrete sources. Mohammad-Djafari =-=[18]-=- derived the posterior in terms of the likelihood function and assigned prior probability density functions (pdf) to sources and the mixing matrix. Independently, in [17], Knuth showed that the blind ... |

18 |
On some Bayesian/ regularization methods for image restoration
- Archer, Titterington
- 1995
(Show Context)
Citation Context ...ocessing in the absence of ground truth information. Several cases of inverse problems can be given. In image de-noising and restoration, the source image is estimated from its noisy observation [1], =-=[2]-=-, [3]. In superresolution image reconstruction and image fusion, the source image is estimated from multiple observations [4], [5]. In image separation multiple source images are extracted from their ... |

18 |
Blind separation of noisy Gaussian stationary sources. Application to cosmic microwave background imaging
- Cardoso, Snoussi, et al.
(Show Context)
Citation Context ... from the true joint pdf, and one can obtain both MAP and Mean Square Error (MSE) estimations of the variables. Another avenue of research in BSS addresses the source modeling problem. Cardoso et al. =-=[25]-=- have used a stationary Gaussian source model and exploited the spectral diversity as the separability criterion. This approach is based on the nonstationarity in Fourier domain, which is the third cr... |

11 |
A Markov model for blind image separation by a mean-field EM algorithm
- Tonazzini, Bedini, et al.
- 2006
(Show Context)
Citation Context ...fier 10.1109/TIP.2009.2012905 In this paper, we address the image source separation problem. Some current applications of image source separation include astrophysical images [6], document processing =-=[7]-=-, and analysis of fMRI data [8]. The classical blind source separation techniques are based on the assumption of source independence and aim to recover unknown sources in the lack of information about... |

9 |
Bayesian source separation with mixture of Gaussians prior for sources and Gaussian prior for mixture coefficients
- Snoussi, Mohammad-Djafari
- 2000
(Show Context)
Citation Context ...arious applications of this strategy on different sources models list as follows. The parameters of mixture of Gaussians have been obtained by the EM method as in Moulines et al. [20], Snoussi et al. =-=[21]-=- and Attias [22], and by variational EM by Miskin [23]. Miskin [23] and Kuruoglu et al. [24] have applied their approach to various image separation problems. For complex models, integration is not tr... |

9 | Blind component separation in wavelet space : Application to CMB analysis
- Moudden, Cardoso, et al.
- 2005
(Show Context)
Citation Context ...ch is based on the nonstationarity in Fourier domain, which is the third criterion of separability according to Cardoso [16]. The wavelet domain extension of this method is proposed by Moudden et al. =-=[26]-=-. The wavelet domain source separation has been also studied in [27] and [28]. Another frequently used iid model is mixture of Gaussian densities [20]–[24]. Hosseini et al. [29] proposed a method to s... |

9 | Allsky astrophysical component separation with fast independent component analysis (FastICA - Maino, Farusi, et al. - 2002 |

7 | A Tonazzini, Separation of correlated astrophysical sources using multiple-lag data covariance matrices
- Bedini, Herranz, et al.
- 2005
(Show Context)
Citation Context ...pproximated by the empirical means of the Markov chains. The components are estimated via a Wiener filter as in [25]. The SOBI approach was applied to correlated astrophysical component separation in =-=[35]-=-. Another 1-D separation method that utilizes fully Bayesian separation is given in [36] where the authors work in the spatial domain avoiding the problems raised by ignoring nonstationarity by workin... |

6 |
Source separation in astrophysical maps using independent factor analysis”, Neural Networks, to appear
- Kuruoglu, Bedini, et al.
- 2003
(Show Context)
Citation Context ...ters of mixture of Gaussians have been obtained by the EM method as in Moulines et al. [20], Snoussi et al. [21] and Attias [22], and by variational EM by Miskin [23]. Miskin [23] and Kuruoglu et al. =-=[24]-=- have applied their approach to various image separation problems. For complex models, integration is not tractable. In the second approach, one uses the joint posterior density of complete variable s... |

5 |
Jojic “A comparison of algorithms for inference and learning in probabilistic graphical models
- Frey, N
- 2005
(Show Context)
Citation Context ...) s t+1 l = max sl p(sl|y1:K, θ t −sl ) (9) A t+1 = max A p(A|y1:K, θ t −A) (10) (σ 2 1:K) t+1 = max σ2 p(Σ|y1:K, θ 1:K t −σ2 ) (11) 1:K The given procedure in (9-11) represents ICM update steps [19],=-=[37]-=-. If the direct solution of this maximization is not possible, iterative optimization methods can be used, such as steepest descent or Newton-Raphson. Difficulties may arise in MAP estimation due to n... |

4 |
Numerical Bayesian Methods Applied to
- Ruanaidh, Fitzgerald
- 1996
(Show Context)
Citation Context ...direct sampling from the joint distribution, p(s1:L, A, σ 2 1:K |y1:K), is not possible. We, therefore, use Gibbs sampler to break down the multivariate sampling problem into a set of univariate ones =-=[38]-=-, [39]. In other words, sampling from conditional distribution of all the unknowns is conducted separately from their corresponding marginal densities. When this iterative procedure converges, then th... |

3 | Blind separation of auto-correlated images from noisy mixtures using MRF models
- Tonazzini, Bedini, et al.
- 2003
(Show Context)
Citation Context ...als. This 1-D treatment of image data precludes the full exploitation of the rich 2-D structure of images. There are 2-D special Bayesian source separation studies in the literature. Tonazzini et al. =-=[30]-=- proposed blind separation of sources using MRF models. In [30], they used a hybrid alternating maximization method in a simulated annealing scheme. The sources are modeled with convex and nonconvex e... |

3 | Source separation in noisy astrophysical images modelled by Markov random fields
- Kuruoglu, Tonazzini, et al.
- 2004
(Show Context)
Citation Context ...e Metropolis method. Snoussi et al. [31] proposed the Gaussian MRF model for source images and implemented a fast MCMC algorithm for joint separation and segmentation of mixed images. Kuruoglu et al. =-=[32]-=- gave a MRF formulation and used it as an edge-preserving regularizer to model the pixel-by-pixel interactions. In their work, Maximum Likelihood (ML) estimation is used for the parameters of the mixi... |

3 |
Fully Bayesian source separation of astrophysical images modelled by a mixture of Gaussians
- Wilson, Kuruoglu, et al.
- 2008
(Show Context)
Citation Context ...a a Wiener filter as in [25]. The SOBI approach was applied to correlated astrophysical component separation in [35]. Another 1-D separation method that utilizes fully Bayesian separation is given in =-=[36]-=- where the authors work in the spatial domain avoiding the problems raised by ignoring nonstationarity by working in the frequency domain. The first attempt to exploit the MRF modeled astrophysical co... |

2 |
Kuruoglu “Image separation using particle filters
- Costagli, and
- 2007
(Show Context)
Citation Context ...r). Digital Object Identifier 10.1109/TIP.2009.2012905 In this paper, we address the image source separation problem. Some current applications of image source separation include astrophysical images =-=[6]-=-, document processing [7], and analysis of fMRI data [8]. The classical blind source separation techniques are based on the assumption of source independence and aim to recover unknown sources in the ... |

1 |
A Bayesian approach to source separation,” presented at the Int
- Mohammad-Djafari
- 1999
(Show Context)
Citation Context ...optimization of all parameters. In [12], Belouchrani and Cardoso considered maximization of likelihood via Expectation-Maximization (EM) algorithm for separation of discrete sources. Mohammad-Djafari =-=[18]-=- derived the posterior in terms of the likelihood function and assigned prior probability density functions (pdf) to sources and the mixing matrix. Independently, in [17], Knuth showed that the blind ... |

1 |
Maximum likelihod for blind separation and deconvolution of noisy signals using mixture models
- Moulines, Cardoso, et al.
- 1997
(Show Context)
Citation Context ...dels are preferred. Various applications of this strategy on different sources models list as follows. The parameters of mixture of Gaussians have been obtained by the EM method as in Moulines et al. =-=[20]-=-, Snoussi et al. [21] and Attias [22], and by variational EM by Miskin [23]. Miskin [23] and Kuruoglu et al. [24] have applied their approach to various image separation problems. For complex models, ... |

1 | Mohammad-Djafari “Bayesian wavelet based signal and image separation
- Ichir, A
- 2003
(Show Context)
Citation Context ...hird criterion of separability according to Cardoso [16]. The wavelet domain extension of this method is proposed by Moudden et al. [26]. The wavelet domain source separation has been also studied in =-=[27]-=- and [28]. Another frequently used iid model is mixture of Gaussian densities [20]–[24]. Hosseini et al. [29] proposed a method to separate sources based on a Markov chain model, so that the time corr... |

1 | Blind source separation with non-stationary mixing using wavelets
- Addison, Roberts
- 2006
(Show Context)
Citation Context ...erion of separability according to Cardoso [16]. The wavelet domain extension of this method is proposed by Moudden et al. [26]. The wavelet domain source separation has been also studied in [27] and =-=[28]-=-. Another frequently used iid model is mixture of Gaussian densities [20]–[24]. Hosseini et al. [29] proposed a method to separate sources based on a Markov chain model, so that the time correlation i... |

1 |
Fast MCMC spectral matching separation in noisy Gaussian mixtures: Application to astrophysics
- Snoussi
- 2006
(Show Context)
Citation Context ...tions apply.984 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 18, NO. 5, MAY 2009 separable in the Fourier domain. A fully Bayesian version of the mixture of noisy Gaussian sources [25] is proposed in =-=[34]-=-. In this version, Gibbs sampling is used for drawing samples of the mixing matrix, noise and sources covariance matrices in the frequency domain. The final estimate of the parameters are found by pos... |

1 |
A MCMC approach for Bayesian super-resolution image reconstruction
- Tian, Ma
- 2005
(Show Context)
Citation Context ... GS-MRF alone requires 73000 iterations (275.81 min). Such a cascade stratagem to reduce convergence time was also applied to inverse image problems solved by complex numerical methods by Tian and Ma =-=[42]-=-. To show the sensitivity of the method to its parameters and , we performed an experiment where they were varied around and values, their optimal setting. We chose the noiseless case of Texture 3 and... |

1 |
Delabrouille and G. Patanchon “Blind separation of noisy Gaussian stationary sources: Application to cosmic microwave background imaging
- Cardoso, Snoussi, et al.
- 2002
(Show Context)
Citation Context ... from the true joint pdf, and one can obtain both MAP and Mean Square Error (MSE) estimations of the variables. Another avenue of research in BSS addresses the source modeling problem. Cardoso et al. =-=[25]-=- have used a stationary Gaussian source model and exploited the spectral diversity as the separability criterion. This approach is based on the nonstationarity in Fourier domain, which is the third cr... |