Computing (ratios of) normalizing constants of probability models is a fundamental computational problem for many statistical and scientific studies. Monte Carlo simulation is an effective technique, especially with complex and high-dimensional models. This paper aims to bring to the attention of general statistical audiences of some effective methods originating from theoretical physics and at the same time to explore these methods from a more statistical perspective, through establishing theoretical connections and illustrating their uses with statistical problems. We show that the acceptance ratio method and thermodynamic integration are natural generalizations of importance sampling, which is most familiar to statistical audiences. The former generalizes importance sampling through the use of a single "bridge" density and is thus a case of bridge sampling in the sense of Meng and Wong (1996). Thermodynamic integration, which is also known in the numerical analysis literature as Oga...

. I present a new Markov chain sampling method appropriate for distributions with isolated modes. Like the recently-developed method of "simulated tempering", the "tempered transition" method uses a series of distributions that interpolate between the distribution of interest and a distribution for which sampling is easier. The new method has the advantage that it does not require approximate values for the normalizing constants of these distributions, which are needed for simulated tempering, and can be tedious to estimate. Simulated tempering performs a random walk along the series of distributions used. In contrast, the tempered transitions of the new method move systematically from the desired distribution, to the easily-sampled distribution, and back to the desired distribution. This systematic movement avoids the inefficiency of a random walk, an advantage that unfortunately is cancelled by an increase in the number of interpolating distributions required. Because of this, the sa...

This work addresses the data--fusion paradigm of multiple targets detected by multiple sensors in the presence of uncertainty. The integrating--information method is developed in a Bayesian framework which allows to embed the data fusion requirements naturally. The merging task is formulated as the minimization of the energy function of a Potts--spin glass where the interactions are given by the degree of consistency of two measurements performed by different sensors. The estimates of the probabilities on which the interactions rely are provided by of a feed--forward neural network. The information is then combined by a self--organizing system which produces a world image by resolving inconsistencies and integrates prior knowledge, such as geometrical constraints and world assumptions. The method is developed for a generic application, but a potential use, based on a specific problem posed by the aircraft industry is presented. This example allows to identify the main features of the m...

In this paper, we discuss image restoration with a Markov field prior that exhibits phase transition. First, we point out that hyperparameter estimation with maximum marginal likelihood procedure is often inadequate for natural images with large clusters (large domains). Next, we will discuss the ways to avoid the di#culty. Specifically, the use of the microcanonical distribution as a prior is useful for this purpose. We also discuss the concept of multicanonical prior and introduce a class of random field models that contains Markov field models as a special case.

I discuss optimized data analysis and Monte Carlo methods. Reweighting methods are discussed through examples, like Lee-Yang zeroes in the Ising model and the absence of deconfinement in QCD. I discuss reweighted data analysis and multi-hystogramming. I introduce Simulated Tempering, and as an example its application to the Random Field Ising Model. I illustrate Parallel Tempering, and discuss some technical crucial details like thermalization and volume scaling. I give a general perspective by discussing Umbrella Methods and the Multicanonical approach. Lectures given at the 1996 Budapest Summer School on Monte Carlo Methods. cond-mat/9612010

We study the 4d Heisenberg spin glass model with Gaussian nearestneighbor interactions. We use finite size scaling to analyze the data. We find a behavior consistent with a finite temperature spin glass transition. Our estimates for the critical exponents agree with the results from "-expansion. PACS Numbers 7510N-0270 Preprint Roma1 n o 1054 The lower critical dimension d l of the short range models remains one of the most controversial questions in the spin glass [1]-[3] theory. In the last ten years a consistent number of works has been devoted to studying the Ising model, which seems to be [4, 5] very close to d l in d = 3. In the case of the short range isotropic Heisenberg spin glass, the conclusions of various computer simulations in d = 3 [6, 7] agree that the system is below d l . Using domain wall renormalization group techniques, it was argued [2, 3] that d l = 4 for this model. To our knowledge, there are no previous numerical simulations on the 4d Heisenberg spin glass...

published in

Abstract. Computing (ratios of) normalizing constants of probability models is a fundamental computational problem for many statistical and scientific studies. Monte Carlo simulation is an effective technique, especially with complex and high-dimensional models. This paper aims to bring to the attention of general statistical audiences of some effective methods originating from theoretical physics and at the same time to explore these methods from a more statistical perspective, through establishing theoretical connections and illustrating their uses with statistical problems. We show that the acceptance ratio method and thermodynamic integration are natural generalizations of importance sampling, which is most familiar to statistical audiences. The former generalizes importance sampling through the use of a single “bridge ” density and is thus a case of bridge sampling in the sense of Meng and Wong. Thermodynamic integration, which is also known in the numerical analysis literature as Ogata’s method for high-dimensional integration, corresponds to the use of infinitely many and continuously connected bridges (and thus a “path”). Our path sampling formulation offers more flexibility and thus potential efficiency to thermodynamic integration, and the search of optimal paths turns out to have close connections with the Jeffreys prior density and the Rao and Hellinger distances between two densities. We provide an informative theoretical example as well as two empirical examples (involving 17- to 70-dimensional integrations) to illustrate the potential and implementation of path sampling. We also discuss some open problems.