Abstract

this paper we investigate the convergence properties of the Gibbs sampler as applied to a particular hierarchical Bayes model. The model is related to James-Stein estimators (James and Stein, 1961; Efron and Morris, 1973, 1975; Morris, 1983). Briefly, James-Stein estimators may be defined as the mean of a certain empirical Bayes posterior distribution (as discussed in the next section). We consider the problem of using the Gibbs sampler as a way of sampling from a richer posterior distribution, as suggested by Jun Liu (personal communication). Such a technique would eliminate the need to estimate a certain parameter empirically and to provide a "guess" at another one, and would give additional information about the distribution of the parameters involved. We consider, in particular, the convergence properties of this Gibbs sampler. For a certain range of prior distributions, we establish (Section 3) rigorous, numerical, reasonable rates of convergence. The bounds are obtained using the methods of Rosenthal (1995b). We thus rigorously bound the running time for this Gibbs sampler to converge to the posterior distribution, within a specified accuracy (as measured by total variation distance). We provide a general formula for this bound, which is of reasonable size, in terms of the prior distribution and the data. This Gibbs sampler is perhaps the most complicated example to date for which reasonable quantitative convergence rates have been obtained. We apply our bounds to the numerical baseball data of Efron and Morris (1975) and Morris (1983), based on batting averages of baseball players, and show that approximately 140 iterations are sufficient to achieve convergence in this case. For a different range of prior distributions, we use the Submartingale Convergence Theo...

Citations

3012	Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images - Geman, Geman - 1984
2262	Convergence of Probability Measures - Billingsley - 1968
1644	Equation of state calculation by fast computing machines - Metropolis, AN, et al. - 1953
886	Monte Carlo sampling methods using Markov chains and their applications - Hastings - 1970
401	Wong W: The calculation of posterior distributions by data augmentation - Tanner - 1987
315	Sampling based approaches to calculating marginal densities - Gelfand, Smith - 1990
257	Approximating the Permanent - Jerrum, Sinclair - 1989
184	Estimation with quadratic loss - James, Stein - 1955
164	Minorization conditions and convergence rates for Markov chain Monte - Rosenthal - 1995
135	Rate of convergences of the Hasting and Metropolis algorithms - Mengersen, Tweedie - 1996
128	Illustration of Bayesian inference in normal data models using Gibbs sampling - Gelfand, Hills, et al. - 1990
77	Computable bounds for convergence rates of Markov chains - Meyn, Tweedie - 1994
73	Regeneration in markov chain samplers - Mykland, Tierney, et al. - 1995
57	Data analysis using Stein's estimator and its generalizations - Efron, CN - 1975
39	On the convergence of successive substitution sampling - Schervish, Carlin - 1992
35	Geometric convergence rates for stochastically ordered Markov chains - Lund, Tweedie - 1994
31	Sampling from log-concave distributions - Frieze, Kannan, et al. - 1994
28	Convergence rates of the Gibbs sampler, the Metropolis algorithm and other single-site updating dynamics - Frigessi, Stefano
25	Convergence diagnostics of the Gibbs sampler - Roberts - 1992
24	Comparing sweep strategies for stochastic relaxation - Amit, Grenander - 1991
23	Stein's estimation rule and its competitors an empirical bayes approach - Efron, Morris - 1973
22	Rates of convergence for data augmentation on finite sample spaces - Rosenthal - 1993
17	Parametric empirical Bayes confidence intervals - Morris - 1983
17	Rates of convergence for Gibbs sampler for variance components models - Rosenthal - 1995
15	Rates of convergence for everywhere-positive Markov chains - Baxter, Rosenthal - 1995
7	Correlation structure and the convergence of the Gibbs sampler, II: Applications to various scans - Liu, Wong, et al. - 1991
4	Prediction in a multivariate normal setting: Coherence and incoherence - Eaton, Sudderth - 1993