Abstract

This article considers Markov chain computational methods for incorporating uncertainty about the dimension of a parameter when performing inference within a Bayesian setting. A general class of methods is proposed for performing such computations, based upon a product space representation of the problem which is similar to that of Carlin and Chib. It is shown that all of the existing algorithms for incorporation of model uncertainty into Markov chain Monte Carlo (MCMC) can be derived as special cases of this general class of methods. In particular, we show that the popular reversible jump method is obtained when a special form of Metropolis--Hastings (M--H) algorithm is applied to the product space. Furthermore, the Gibbs sampling method and the variable selection method are shown to derive straightforwardly from the general framework. We believe that these new relationships between methods, which were until now seen as diverse procedures, are an important aid to the understanding of MCMC model selection procedures and may assist in the future development of improved procedures. Our discussion also sheds some light upon the important issues of "pseudo-prior" selection in the case of the Carlin and Chib sampler and choice of proposal distribution in the case of reversible jump. Finally, we propose efficient reversible jump proposal schemes that take advantage of any analytic structure that may be present in the model. These proposal schemes are compared with a standard reversible jump scheme for the problem of model order uncertainty in autoregressive time series, demonstrating the improvements which can be achieved through careful choice of proposals

Citations

3012	Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images - Geman, Geman - 1984
1644	Equation of state calculation by fast computing machines - Metropolis, AN, et al. - 1953
886	Monte Carlo sampling methods using Markov chains and their applications - Hastings - 1970
589	AFM. Sampling-based approaches to calculating marginal densities - AE, Smith - 1990
588	Monte Carlo statistical methods - Robert, Casella - 2004
578	PJ: Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika - Green - 1995
445	Inference from iterative simulation using multiple sequences (with discussion). Stat Sci - Gelman, DB - 1992
325	PJ: On Bayesian analysis of mixtures with an unknown number of components (with discussion - Richardson, Green - 1997
231	Marginal likelihood from the Gibbs output - Chib - 1995
230	RE: Variable selection via Gibbs sampling - EI, McCulloch - 1993
168	Bayesian graphical models for discrete data - Madigan, York - 1995
133	Hoeting J: Bayesian model averaging for linear regression models - Raftery, Madigan - 1997
120	Bayesian model choice via Markov chain Monte Carlo - Carlin, Chib - 1995
64	Automatic Bayesian curve fitting - Denison, Mallick, et al. - 1998
64	Bayesian model comparison via jump diffusions - Phillips, Smith - 1996
57	Analysis of Multivariate Probit Models - Chib, Greenberg - 1998
48	Variable selection and model comparison in regression - Geweke - 1996
46	Markov chain Monte Carlo in conditionally Gaussian state space models - Carter, Kohn - 1996
38	Prediction via Orthogonalized Model Mixing - Clyde, DeSimone, et al. - 1996
36	Bayesian analysis of autoregressive time series via the Gibbs sampler - McCulloch, Tsay - 1994
34	On Bayesian model and variable selection using MCMC - Dellaportas, Forster, et al.
32	Variable selection for regression models - Kuo, Mallick - 1998
22	Bayesian estimation of an autoregressive model using Markov chain Monte Carlo - Barnett, Kohn, et al. - 1996
19	A reversible jump sampler for autoregressive time series - Troughton, Godsill - 1998
18	A method for simultaneous variable selection and outlier identification in linear regression - Hoeting, Raftery, et al. - 1996
16	Bayesian enhancement of speech and audio signals which can be modelled as ARMA processes - Godsill - 1997
12	A reversible jump MCMC sampler for Bayesian analysis of ARMA time series - Barbieri, O’Hagan - 1996
10	Convergence assessment for Reversible Jump MCMC Simulations," Bayesian Statistics 6 - Brooks, Giudici - 1998
9	Jump-diffusion processes for abduction and recognition of biological shapes - Grenander, Miller - 1991
9	A sampling based approach to line scratch removal from motion picture frames - Morris, Fitzgerald, et al. - 1996
9	Hypothesis Testing and Model Selection,” Markov Chain Monte Carlo in Practice - Raftery - 1996
5	Multiple-Order Markov Chain Monte Carlo Sampling Methods with Application to a Changepoint Model - Stark, Fitzgerald, et al. - 1997
4	Unsupervised Image Segmentation - Barker, Rayner - 1998
4	Carlin and Chib do not need to sample from pseudopriors - Green, O’Hagan - 1998
1	A Candidate’s Formula—A Curious Result - Besag - 1989
1	Discussion of “Bayesian Analysis of Mixtures With an Unknown Number of Components,” by - GODSILL - 1997
1	Robust Treatment of Impulsive Noise - Godsill, Rayner - 1996
1	Priors and Component Structures in Autoregressive Time Series Models - METHODS - 1999