Results 1 
8 of
8
Simulating Normalized Constants: From Importance Sampling to Bridge Sampling to Path Sampling
 Statistical Science, 13, 163–185. COMPARISON OF METHODS FOR COMPUTING BAYES FACTORS 435
, 1998
"... Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at ..."
Abstract

Cited by 146 (4 self)
 Add to MetaCart
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
Simulating ratios of normalizing constants via a simple identity: A theoretical exploration
 Statistica Sinica
, 1996
"... Abstract: Let pi(w),i =1, 2, be two densities with common support where each density is known up to a normalizing constant: pi(w) =qi(w)/ci. We have draws from each density (e.g., via Markov chain Monte Carlo), and we want to use these draws to simulate the ratio of the normalizing constants, c1/c2. ..."
Abstract

Cited by 109 (4 self)
 Add to MetaCart
Abstract: Let pi(w),i =1, 2, be two densities with common support where each density is known up to a normalizing constant: pi(w) =qi(w)/ci. We have draws from each density (e.g., via Markov chain Monte Carlo), and we want to use these draws to simulate the ratio of the normalizing constants, c1/c2. Such a computational problem is often encountered in likelihood and Bayesian inference, and arises in fields such as physics and genetics. Many methods proposed in statistical and other literature (e.g., computational physics) for dealing with this problem are based on various special cases of the following simple identity: c1 c2 = E2[q1(w)α(w)] E1[q2(w)α(w)]. Here Ei denotes the expectation with respect to pi (i =1, 2), and α is an arbitrary function such that the denominator is nonzero. A main purpose of this paper is to provide a theoretical study of the usefulness of this identity, with focus on (asymptotically) optimal and practical choices of α. Using a simple but informative example, we demonstrate that with sensible (not necessarily optimal) choices of α, we can reduce the simulation error by orders of magnitude when compared to the conventional importance sampling method, which corresponds to α =1/q2. We also introduce several generalizations of this identity for handling more complicated settings (e.g., estimating several ratios simultaneously) and pose several open problems that appear to have practical as well as theoretical value. Furthermore, we discuss related theoretical and empirical work.
Blocking Gibbs Sampling for Linkage Analysis in Large Pedigrees with Many Loops
 AMERICAN JOURNAL OF HUMAN GENETICS
, 1996
"... We will apply the method of blocking Gibbs sampling to a problem of great importance and complexity  linkage analysis. Blocking Gibbs combines exact local computations with Gibbs sampling in a way that complements the strengths of both. The method is able to handle problems with very high complexi ..."
Abstract

Cited by 24 (2 self)
 Add to MetaCart
We will apply the method of blocking Gibbs sampling to a problem of great importance and complexity  linkage analysis. Blocking Gibbs combines exact local computations with Gibbs sampling in a way that complements the strengths of both. The method is able to handle problems with very high complexity such as linkage analysis in large pedigrees with many loops; a task that no other known method is able to handle. New developments of the method are outlined, and it is applied to a highly complex linkage problem.
Semiparametric Bayesian Analysis Of Survival Data
 Journal of the American Statistical Association
, 1996
"... this paper are motivated and aimed at analyzing some common types of survival data from different medical studies. We will center our attention to the following topics. ..."
Abstract

Cited by 23 (0 self)
 Add to MetaCart
this paper are motivated and aimed at analyzing some common types of survival data from different medical studies. We will center our attention to the following topics.
Estimating Ratios of Normalizing Constants for Densities with Different Dimensions
 STATISTICA SINICA
, 1997
"... In Bayesian inference, a Bayes factor is defined as the ratio of posterior odds versus prior odds where posterior odds is simply a ratio of the normalizing constants of two posterior densities. In many practical problems, the two posteriors have different dimensions. For such cases, the current Mont ..."
Abstract

Cited by 17 (2 self)
 Add to MetaCart
In Bayesian inference, a Bayes factor is defined as the ratio of posterior odds versus prior odds where posterior odds is simply a ratio of the normalizing constants of two posterior densities. In many practical problems, the two posteriors have different dimensions. For such cases, the current Monte Carlo methods such as the bridge sampling method (Meng and Wong 1996), the path sampling method (Gelman and Meng 1994), and the ratio importance sampling method (Chen and Shao 1994) cannot directly be applied. In this article, we extend importance sampling, bridge sampling, and ratio importance sampling to problems of different dimensions. Then we find global optimal importance sampling, bridge sampling, and ratio importance sampling in the sense of minimizing asymptotic relative meansquare errors of estimators. Implementation algorithms, which can asymptotically achieve the optimal simulation errors, are developed and two illustrative examples are also provided.
Learning and evaluating Boltzmann machines
, 2008
"... We provide a brief overview of the variational framework for obtaining deterministic approximations or upper bounds for the logpartition function. We also review some of the Monte Carlo based methods for estimating partition functions of arbitrary Markov Random Fields. We then develop an annealed i ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
We provide a brief overview of the variational framework for obtaining deterministic approximations or upper bounds for the logpartition function. We also review some of the Monte Carlo based methods for estimating partition functions of arbitrary Markov Random Fields. We then develop an annealed importance sampling (AIS) procedure for estimating partition functions of restricted Boltzmann machines (RBM’s), semirestricted Boltzmann machines (SRBM’s), and Boltzmann machines (BM’s). Our empirical results indicate that the AIS procedure provides much better estimates of the partition function than some of the popular variationalbased methods. Finally, we develop a new learning algorithm for training general Boltzmann machines and show that it can be successfully applied to learning good generative models. Learning and Evaluating Boltzmann Machines
Monte Carlo Methods on Bayesian Analysis of Constrained Parameter Problems with Normalizing Constants
 Biometrika
, 1998
"... Constraints on the parameters in a Bayesian hierarchical model typically make Bayesian computation and analysis complicated. As Gelfand, Smith and Lee (1992) remarked, it is almost impossible to sample from a posterior distribution when its density contains analytically intractable integrals (normal ..."
Abstract

Cited by 8 (3 self)
 Add to MetaCart
Constraints on the parameters in a Bayesian hierarchical model typically make Bayesian computation and analysis complicated. As Gelfand, Smith and Lee (1992) remarked, it is almost impossible to sample from a posterior distribution when its density contains analytically intractable integrals (normalizing constants) that depend on the (hyper) parameters. Therefore, the Gibbs sampler or the Metropolis algorithm can not be directly applied to such problems. In this paper, using the idea of "reweighting mixtures" of Geyer (1994), we develop alternative Monte Carlo based methods to determine properties of the desired Bayesian posterior distribution. Necessary theory and two illustrative examples are provided. Keywords and Phrases: Bayesian computation; Bayesian hierarchical model; Gibbs sampler; Markov chain Monte Carlo; Marginal posterior density estimation; Posterior distribution; Sensitivity of prior specification. 1 Introduction In this article we consider a Bayesian hierarchical mod...
NORMALIZING CONSTANTS
"... Abstract. Computing (ratios of) normalizing constants of probability models is a fundamental computational problem for many statistical and scientific studies. Monte Carlo simulation is an effective technique, especially with complex and highdimensional models. This paper aims to bring to the atten ..."
Abstract
 Add to MetaCart
Abstract. Computing (ratios of) normalizing constants of probability models is a fundamental computational problem for many statistical and scientific studies. Monte Carlo simulation is an effective technique, especially with complex and highdimensional models. This paper aims to bring to the attention of general statistical audiences of some effective methods originating from theoretical physics and at the same time to explore these methods from a more statistical perspective, through establishing theoretical connections and illustrating their uses with statistical problems. We show that the acceptance ratio method and thermodynamic integration are natural generalizations of importance sampling, which is most familiar to statistical audiences. The former generalizes importance sampling through the use of a single “bridge ” density and is thus a case of bridge sampling in the sense of Meng and Wong. Thermodynamic integration, which is also known in the numerical analysis literature as Ogata’s method for highdimensional integration, corresponds to the use of infinitely many and continuously connected bridges (and thus a “path”). Our path sampling formulation offers more flexibility and thus potential efficiency to thermodynamic integration, and the search of optimal paths turns out to have close connections with the Jeffreys prior density and the Rao and Hellinger distances between two densities. We provide an informative theoretical example as well as two empirical examples (involving 17 to 70dimensional integrations) to illustrate the potential and implementation of path sampling. We also discuss some open problems.