Results 1 
8 of
8
Simulating Normalized Constants: From Importance Sampling to Bridge Sampling to Path Sampling
 Statistical Science, 13, 163–185. COMPARISON OF METHODS FOR COMPUTING BAYES FACTORS 435
, 1998
"... Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at ..."
Abstract

Cited by 146 (4 self)
 Add to MetaCart
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
Simulating ratios of normalizing constants via a simple identity: A theoretical exploration
 Statistica Sinica
, 1996
"... Abstract: Let pi(w),i =1, 2, be two densities with common support where each density is known up to a normalizing constant: pi(w) =qi(w)/ci. We have draws from each density (e.g., via Markov chain Monte Carlo), and we want to use these draws to simulate the ratio of the normalizing constants, c1/c2. ..."
Abstract

Cited by 109 (4 self)
 Add to MetaCart
Abstract: Let pi(w),i =1, 2, be two densities with common support where each density is known up to a normalizing constant: pi(w) =qi(w)/ci. We have draws from each density (e.g., via Markov chain Monte Carlo), and we want to use these draws to simulate the ratio of the normalizing constants, c1/c2. Such a computational problem is often encountered in likelihood and Bayesian inference, and arises in fields such as physics and genetics. Many methods proposed in statistical and other literature (e.g., computational physics) for dealing with this problem are based on various special cases of the following simple identity: c1 c2 = E2[q1(w)α(w)] E1[q2(w)α(w)]. Here Ei denotes the expectation with respect to pi (i =1, 2), and α is an arbitrary function such that the denominator is nonzero. A main purpose of this paper is to provide a theoretical study of the usefulness of this identity, with focus on (asymptotically) optimal and practical choices of α. Using a simple but informative example, we demonstrate that with sensible (not necessarily optimal) choices of α, we can reduce the simulation error by orders of magnitude when compared to the conventional importance sampling method, which corresponds to α =1/q2. We also introduce several generalizations of this identity for handling more complicated settings (e.g., estimating several ratios simultaneously) and pose several open problems that appear to have practical as well as theoretical value. Furthermore, we discuss related theoretical and empirical work.
ML parameter estimation for Markov random fields, with applications to Bayesian tomography
 IEEE Trans. on Image Processing
, 1998
"... Abstract 1 Markov random fields (MRF) have been widely used to model images in Bayesian frameworks for image reconstruction and restoration. Typically, these MRF models have parameters that allow the prior model to be adjusted for best performance. However, optimal estimation of these parameters (so ..."
Abstract

Cited by 49 (18 self)
 Add to MetaCart
Abstract 1 Markov random fields (MRF) have been widely used to model images in Bayesian frameworks for image reconstruction and restoration. Typically, these MRF models have parameters that allow the prior model to be adjusted for best performance. However, optimal estimation of these parameters (sometimes referred to as hyperparameters) is difficult in practice for two reasons: 1) Direct parameter estimation for MRF’s is known to be mathematically and numerically challenging. 2) Parameters can not be directly estimated because the true image crosssection is unavailable. In this paper, we propose a computationally efficient scheme to address both these difficulties for a general class of MRF models, and we derive specific methods of parameter estimation for the MRF model known as a generalized Gaussian MRF (GGMRF). The first section of the paper derives methods of direct estimation of scale and shape parameters for a general continuously valued MRF. For the GGMRF case, we show that the ML estimate of the scale parameter, σ, has a simple closed form solution, and we present an efficient scheme for computing the ML estimate of the shape parameter, p, by an offline numerical computation of the dependence of the partition function on p.
Stochastic approximation algorithms for partition function estimation of Gibbs random fields
 IEEE Trans. Inform. Theory
, 1997
"... Abstract—We present an analysis of recently proposed Monte Carlo algorithms for estimating the partition function of a Gibbs random field. We show that this problem reduces to estimating one or more expectations of suitable functionals of the Gibbs states with respect to properly chosen Gibbs distri ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
Abstract—We present an analysis of recently proposed Monte Carlo algorithms for estimating the partition function of a Gibbs random field. We show that this problem reduces to estimating one or more expectations of suitable functionals of the Gibbs states with respect to properly chosen Gibbs distributions. As expected, the resulting estimators are consistent. Certain generalizations are also provided. We study computational complexity with respect to grid size and show that Monte Carlo partition function estimation algorithms can be classified into two categories: EType algorithms that are of exponential complexity and PType algorithms that are of polynomial complexity, Turing reducible to the problem of sampling from the Gibbs distribution. EType algorithms require estimating a single expectation, whereas, PType algorithms require estimating a number of expectations with respect to Gibbs distributions which are chosen to be sufficiently “close ” to each other. In the latter case, the required number of expectations is of polynomial order with respect to grid size. We compare computational complexity by using both theoretical results and simulation experiments. We determine the most efficient EType and PType algorithms and conclude that PType algorithms are more appropriate for partition function estimation. We finally suggest a practical and efficient PType algorithm for this task. Index Terms—Computational complexity, Gibbs random fields, importance sampling, Monte Carlo simulations, partition function estimation, stochastic approximation. I.
Robust Priors for Smoothing and Image Restoration
, 1993
"... The Bayesian method for restoring an image corrupted by added Gaussian noise uses a Gibbs prior for the unknown clean image. The potential of this Gibbs prior penalizes differences between adjacent grey levels. In this paper we discuss the choice of the form and the parameters of the penalizing ..."
Abstract
 Add to MetaCart
The Bayesian method for restoring an image corrupted by added Gaussian noise uses a Gibbs prior for the unknown clean image. The potential of this Gibbs prior penalizes differences between adjacent grey levels. In this paper we discuss the choice of the form and the parameters of the penalizing potential in a particular example used previously by Ogata (1990). In this example the clean image is piecewise constant, but the constant patches and the step sizes at edges are small compared with the noise variance. We find that contrary to results reported in Ogata (1990) the Bayesian method performs well provided the potential increases more slowly than a quadratic one and the scale parameter of the potential is sufficiently small. Convex potentials with bounded derivatives perform not much worse than bounded potentials, but are computationally much simpler. For bounded potentials we use a variant of simulated annealling. For quadratic potentials datadriven choices of the smo...
NORMALIZING CONSTANTS
"... Abstract. Computing (ratios of) normalizing constants of probability models is a fundamental computational problem for many statistical and scientific studies. Monte Carlo simulation is an effective technique, especially with complex and highdimensional models. This paper aims to bring to the atten ..."
Abstract
 Add to MetaCart
Abstract. Computing (ratios of) normalizing constants of probability models is a fundamental computational problem for many statistical and scientific studies. Monte Carlo simulation is an effective technique, especially with complex and highdimensional models. This paper aims to bring to the attention of general statistical audiences of some effective methods originating from theoretical physics and at the same time to explore these methods from a more statistical perspective, through establishing theoretical connections and illustrating their uses with statistical problems. We show that the acceptance ratio method and thermodynamic integration are natural generalizations of importance sampling, which is most familiar to statistical audiences. The former generalizes importance sampling through the use of a single “bridge ” density and is thus a case of bridge sampling in the sense of Meng and Wong. Thermodynamic integration, which is also known in the numerical analysis literature as Ogata’s method for highdimensional integration, corresponds to the use of infinitely many and continuously connected bridges (and thus a “path”). Our path sampling formulation offers more flexibility and thus potential efficiency to thermodynamic integration, and the search of optimal paths turns out to have close connections with the Jeffreys prior density and the Rao and Hellinger distances between two densities. We provide an informative theoretical example as well as two empirical examples (involving 17 to 70dimensional integrations) to illustrate the potential and implementation of path sampling. We also discuss some open problems.
der LudwigMaximiliansUniversität München vorgelegt von
"... Arbeit in meinem persönlichen Wohlfühltempo anzufertigen, und jederzeit für Nachfragen zur Verfügung stand. In gleichem Maße gilt mein Dank Leonhard Held, der mir die Bayesianische Seite der Statistik nahe brachte und verständlich machte und stets viel Vertrauen in meine Forschungstätigkeit setzte ( ..."
Abstract
 Add to MetaCart
Arbeit in meinem persönlichen Wohlfühltempo anzufertigen, und jederzeit für Nachfragen zur Verfügung stand. In gleichem Maße gilt mein Dank Leonhard Held, der mir die Bayesianische Seite der Statistik nahe brachte und verständlich machte und stets viel Vertrauen in meine Forschungstätigkeit setzte (“Du bist jetzt ein Selbstläufer”). Schließlich möchte ich mich bei Katja Ickstadt bedanken, die freundlicherweise und trotz eines engen “Terminplans ” die Begutachtung meiner Dissertation übernahm. Von meinen Mitdoktoranden möchte ich zwei ganz besonders hervorheben. Zum einen Leyre Osuna, die mir stets geduldig Nachilfe gab bei allen schwierigen (und auch nicht so schwierigen) MathematikProblemen, mich mit Kaffee versorgte (falls erwünscht) und auch anderweitig für Ablenkung sorgte. Zum anderen wären weite Teile dieser Arbeit sehr lückenhaft geblieben, wenn nicht Volker “Markov Random Man ” Schmid im Zimmer neben mir gesessen hätte. Sein geduldiges Wiederkäuen aller Details über Markov Random Fields, die ich nie verstand und wohl nie verstehen werde, war mehr als hilfreich. Die endgültige Fassung dieser Arbeit hat wesentlich von den englischen Sprachkenntnissen von Manuela Glaser und der Hilfsbereitschaft meines Zimmergenossen Thomas Kneib profitiert, denen ich für ihren Beitrag danke.