This paper is also the originator of the Markov Chain Monte Carlo methods developed in the following chapters. The potential of these two simultaneous innovations has been discovered much latter by statisticians (Hastings 1970; Geman and Geman 1984) than by of physicists (see also Kirkpatrick et al. 1983). 5.5.5 ] PROBLEMS 211

This article considers Markov chain computational methods for incorporating uncertainty about the dimension of a parameter when performing inference within a Bayesian setting. A general class of methods is proposed for performing such computations, based upon a product space representation of the problem which is similar to that of Carlin and Chib. It is shown that all of the existing algorithms for incorporation of model uncertainty into Markov chain Monte Carlo (MCMC) can be derived as special cases of this general class of methods. In particular, we show that the popular reversible jump method is obtained when a special form of Metropolis--Hastings (M--H) algorithm is applied to the product space. Furthermore, the Gibbs sampling method and the variable selection method are shown to derive straightforwardly from the general framework. We believe that these new relationships between methods, which were until now seen as diverse procedures, are an important aid to the understanding of MCMC model selection procedures and may assist in the future development of improved procedures. Our discussion also sheds some light upon the important issues of "pseudo-prior" selection in the case of the Carlin and Chib sampler and choice of proposal distribution in the case of reversible jump. Finally, we propose efficient reversible jump proposal schemes that take advantage of any analytic structure that may be present in the model. These proposal schemes are compared with a standard reversible jump scheme for the problem of model order uncertainty in autoregressive time series, demonstrating the improvements which can be achieved through careful choice of proposals

Introduction When fitting an autoregressive model to Gaussian time series data, often the correct order of the model is unknown. The model order cannot be estimated analytically by conventional Bayesian techniques when the excitation variance is unknown. We present MCMC methods for drawing samples from the joint posterior of all the unknowns, from which Monte Carlo estimates of the quantities of interest can be made, with the possibility of model mixing, if required, for tasks such as prediction, interpolation, smoothing or noise reduction. Previous work on MCMC autoregressive model selection has parameterised the model using partial correlation coefficients (Barnett, Kohn & Sheather 1996, Barbieri & O'Hagan 1996) or pole positions 1 (Huerta & West 1997). These have a simple physical interpretation for certain types of signal, and allow stationarity to be enforced in a straightforward manner. We use the AR parameters, a, directly. This allows

Abstract not found

We explore some aspects of the analysis of latent component structure in non-stationary time series based on time-varying autoregressive (TVAR) models that incorporate uncertainty on model order. Our modelling approach assumes that the AR coecients evolve in time according to a random walk and that the model order may also change in time following a discrete random walk. In addition, we use a conjugate prior structure on the autoregressive coecients and a discrete uniform prior on model order. Simulation from the posterior distribution of the model parameters can be obtained via standard Forward Filtering Backward Simulation algorithms. Aspects of implementation and inference on decompositions, latent structure and model order are discussed for a synthetic series and for an electroencephalogram (EEG) trace previously analysed using xed order TVAR models. Keywords: Dynamic linear models; Time-varying autoregressions; Model uncertainty; Time series decompositions; Markov cha...

This paper poses the problem of model order determination of an autoregressive (AR) process within a Bayesian framework. Several original hierarchical prior models are proposed that allow for the stability of the model to be enforced and account for a possible unknown initial state. Obtaining the posterior model order probabilities requires integration of the resulting posterior distribution, an operation which is analytically intractable. Here stochastic reversible jump Markov chain Monte Carlo (MCMC) algorithms are developed to perform the required integration by simulating from the posterior distribution. The methods developed are evaluated in simulation studies on a number of synthetic and real data sets, and compared to standard model selection criteria. 1 I Introduction When tting an autoregressive (AR) model to real data, e.g. speech, the correct model order is often unknown. Using too low a value for the model order would lead to an undesired smoothing of the data, whereas o...

We review the class of time-varying autoregressive (TVAR) models and a range of related recent developments of Bayesian time series modelling. Beginning with TVAR models in a Bayesian dynamic linear modelling framework, we review aspects of latent structure analysis, including time-domain decomposition methods that provide inferences on the structure underlying non-stationary time series, and that are now central tools in the time series analyst's toolkit. Recent model extensions that deal with model order uncertainty, and are enabled using efficient Markov Chain Monte Carlo simulation methods, are discussed, as are novel approaches to sequential filtering and smoothing using particulate filtering methods. We emphasize the relevance of TVAR modelling in a range of applied contexts, including biomedical signal processing and communications, and highlight some of the central developments via examples arising in studies of multiple electroencephalographic (EEG) traces in neurophysiology. We conclude with comments about current research frontiers.

... efficient proposal schemes for reversible jump MCMC in the context of autoregressive moving average models. In particular, the full conditional distribution is not available for the added parameters and approximations to it are provided by suggesting an adaptive updating scheme which automatically selects proposal parameter values to improve the efficiency of between-model moves. The performance of the proposed algorithms is assessed by simulation studies and the methodology is illustrated by applying it to a real data set.

efficient proposal schemes for reversible jump MCMC in the context of autoregressive moving average models. In particular, the full conditional distribution is not available for the added parameters and approximations to it are provided by suggesting an adaptive updating scheme which automatically selects proposal parameter values to improve the efficiency of between-model moves. The performance of the proposed algorithms is assessed by simulation studies and the methodology is illustrated by applying it to a real data set.

This paper analyzes the Brazilian industrial production index using a Bayesian methodology based on a new class of prior distributions for AR models that is extended to allow for heavy-tailed errors. The analysis shows how a unified approach is able to deal with model uncertainty, inference on latent structure, inference on unitary roots, forecasting and outliers, all at once. This type of modeling avoids the imposition of trends and polynomial seasonal components to capture structure and multiple significant tests to show the presence of an underlying stochastic trend. It also avoids the use of "ad-hoc" diagnostic tools to detect outlying observations by dealing with scale-mixtures of Normals.