This paper is also the originator of the Markov Chain Monte Carlo methods developed in the following chapters. The potential of these two simultaneous innovations has been discovered much latter by statisticians (Hastings 1970; Geman and Geman 1984) than by of physicists (see also Kirkpatrick et al. 1983). 5.5.5 ] PROBLEMS 211

We consider a novel problem of learning an optimal matching, in an online fashion, between two feature spaces that are organized as taxonomies. We formulate this as a multi-armed bandit problem where the arms of the bandit are dependent due to the structure induced by the taxonomies. We then propose a multi-stage hierarchical allocation scheme that improves the explore/exploit properties of the classical multiarmed bandit policies in this scenario. In particular, our scheme uses the taxonomy structure and performs shrinkage estimation in a Bayesian framework to exploit dependencies among the arms, thereby enhancing exploration without losing efficiency on short term exploitation. We prove that our scheme asymptotically converges to the optimal matching. We conduct extensive experiments on real data to illustrate the efficacy of our scheme in practice. 1

Consider the conditionally independent hierarchical model (CIHM) where observations y i are independently distributed from f(y i j` i ), the parameters ` i are independently distributed from distributions g(`j), and the hyperparameters are distributed according to a distribution h(). The posterior distribution of all parameters of the CIHM can be efficiently simulated by Monte Carlo Markov Chain (MCMC) algorithms. Although these simulation algorithms have facilitated the application of CIHM's, they generally have not addressed the problem of computing quantities useful in model selection. This paper explores how MCMC simulation algorithms and other related computational algorithms can be used to compute Bayes factors that are useful in criticizing a particular CIHM. In the case where the CIHM models a belief that the parameters are exchangeable or lie on a regression surface, the Bayes factor can measure the consistency of the data with the structural prior belief. Bayes factors can ...

Consider the exchangeable Bayesian hierarchical model where observations y i are independently distributed from sampling densities with unknown means, the means ¯ i are a random sample from a distribution g, and the parameters of g are assigned a known distribution h. A simple algorithm is presented for summarizing the posterior distribution based on Gibbs sampling and the Metropolis algorithm. The software program Matlab is used to implement the algorithm and provide a graphical output analysis. An binomial example is used to illustrate the flexibility of modeling possible using this algorithm. Methods of model checking and extensions to hierarchical regression modeling are discussed.

ler. Statist. Comput. 1, 105--117. Vines, S.K. and Gilks, W.R. (1994) Reparameterising random interactions for Gibbs sampling. Tech. report, MRC Biostatistics Unit, Cambridge. Vines, S.K., Gilks, W.R. and Wild, P. (1995) Fitting multiple random effect models. Tech. report, Medical Research Council Biostatistics Unit, Institute of Public Health, Cambridge University. Von Neumann, J. (1951) Various techniques used in connection with random digits. J. Resources of the National Bureau of Standards -- Applied Mathematics Series 12, 36--38. Wahba, G. (1981) Spline interpolation and smoothing on the sphere. SIAMSSC 2, 5--16. Wakefield, J.C., Gelfand, A.E. and Smith, A.F.M. (1991) Efficient generation of random variates via the ratio-of-uniforms method. Statistics and Computing 1, 129--33. Wakefield, J.C., Smith, A.F.M., Racine-Poon, A. and Gelfand, A.E. (1994) Bayesian analysis of linear and non-