The need to explore model uncertainty in linear regression models with many predictors has motivated improvements in Markov chain Monte Carlo sampling algorithms for Bayesian variable selection. Currently used sampling algorithms for Bayesian variable selection may perform poorly when there are severe multicollinearities among the predictors. This article describes a new sampling method based on an analogy with the Swendsen-Wang algorithm for the Ising model, and which can give substantial improvements over alternative sampling schemes in the presence of multicollinearity. In linear regression with a given set of potential predictors we can index possible models by a binary parameter vector that indicates which of the predictors are included or excluded. By thinking of the posterior distribution of this parameter as a binary spatial field, we can use auxiliary variable methods inspired by the Swendsen-Wang algorithm for the Ising model to sample from the posterior where dependence among parameters is reduced by conditioning on auxiliary variables. Performance of the method is described for both simulated and real data.

This paper shows (i) improvements over state-of-the-art local feature recognition systems, (ii) how to formulate principled models for automatic local feature selection in object class recognition when there is little supervised data, and (iii) how to formulate sensible spatial image context models using a conditional random field for integrating local features and segmentation cues (superpixels). By adopting sparse kernel methods, Bayesian learning techniques and data association with constraints, the proposed model identifies the most relevant sets of local features for recognizing object classes, achieves performance comparable to the fully supervised setting, and obtains excellent results for image classification.

Generalized additive models (GAM) for modeling nonlinear effects of continuous covariates are now well established tools for the applied statistician. A Bayesian version of GAM’s and extensions to generalized structured additive regression (STAR) are developed. One or two dimensional P-splines are used as the main building block. Inference relies on Markov chain Monte Carlo (MCMC) simulation techniques, and is either based on iteratively weighted least squares (IWLS) proposals or on latent utility representations of (multi)categorical regression models. The approach covers the most common univariate response distributions, e.g. the binomial, Poisson or gamma distribution, as well as multicategorical responses. For the first time, Bayesian semiparametric inference for the widely used multinomial logit model is presented. Two applications on the forest health status of trees and a space-time analysis of health insurance data demonstrate the potential of the approach for realistic modeling of complex problems. Software for the methodology is provided within the public domain package BayesX. Key words: geoadditive models, IWLS proposals, multicategorical response, structured additive predictors, surface smoothing

We consider the input variable selection in complex Bayesian hierarchical models. Our goal is to find a model with the smallest number of input variables having statistically or practically at least the same expected utility as the full model with all the available inputs. A good estimate for the expected utility can be computed using cross-validation predictive densities. In the case of input selection and a large number of input combinations, the computation of the cross-validation predictive densities for each model easily becomes computationally prohibitive. We propose to use the posterior probabilities obtained via variable dimension MCMC methods to find out potentially useful input combinations, for which the final model choice and assessment is done using the expected utilities.

The Bayesian linear model framework has become increasingly popular building block in regression problems. It has been shown to produce models with good predictive power and can be used with basis functions that are nonlinear in the data to provide exible estimated regression functions. Further, model uncertainty can be accounted for by Bayesian model averaging. We propose a more simple way to account for model uncertainty that is based on generalized ridge regression estimators. This is shown to predict well and to be much more computationally ecient than standard model averaging methods. Further, we demonstrate how to eciently mix over dierent sets of basis functions, letting the data determine which are most appropriate for the problem at hand. Keywords: Bayesian model averaging, generalized ridge regression, prediction, regression splines, shrinkage. 1

Copulas have proven to be very successful tools for the flexible modelling of cross-sectional dependence. In this paper we express the dependence structure of continuous time series data using a sequence of bivariate copulas. This corresponds to a type of decomposition recently called a ‘vine ’ in the graphical models literature, where each copula is entitled a ‘pair-copula’. We propose a Bayesian approach for the estimation of this dependence structure for longitudinal data. Bayesian selection ideas are used to identify any independence pair-copulas, with the end result being a parsimonious representation of a time-inhomogeneous Markov process of varying order. Estimates are Bayesian model averages over the distribution of the lag structure of the Markov process. Overall, the pair-copula construction is very general and the Bayesian approach generalises many previous methods for the analysis of longitudinal data. Both the reliability of the proposed Bayesian methodology, and the advantages of the pair-copula formulation, are demonstrated via simulation and two examples. The first is an agricultural science example, while the second is an econometric model for the forecasting of intraday electricity load. For both examples the Bayesian pair-copula model is substantially more flexible than longitudinal models employed previously.

In presenting this thesis in partial fulfilment of the requirements for an advanced

We consider the problem of input variable selection of a Bayesian model. With suitable priors it is possible to have a large number of input variables in Bayesian models, as less relevant inputs can have a smaller effect in the model. To make the model more explainable and easier to analyse, or to reduce the cost of making measurements or the cost of computation, it may be useful to select a smaller set of input variables. Our goal is to find a model with the smallest number of input variables having statistically or practically the same expected utility as the full model. A good estimate for the expected utility, with any desired utility, can be computed using cross-validation predictive densities (Vehtari and Lampinen, 2001). In the case of input selection, there are 2 K input combinations and computing the cross-validation predictive densities for each model easily becomes computationally prohibitive. We propose to use the reversible jump Markov chain Monte Carlo (RJMCMC) method to find out potentially useful input combinations, for which the final model choice and assessment is done using the cross-validation predictive densities. The RJMCMC visits the models according to their posterior probabilities. As models with negligible probability are probably not visited in finite time, the computational savings can be considerable compared to going through all possible models. The posterior probabilities of the models, given by the RJMCMC, are proportional to the product of the prior probabilities of the models and the prior predictive likelihoods of the models. The prior predictive likelihood measures the goodness of the model if no training data were used, and thus can be used to estimate the lower limit of the expected predictive likelihood. These estimates indicate ...

Abstract not found

We consider model selection as a decision problem from a predictive perspective. The optimal Bayesian way of handling model uncertainty is to integrate over model space. Model selection can then be seen as point estimation in the model space. We propose a model selection method based on Kullback-Leibler divergence from the predictive distribution of the full model to the predictive distributions of the submodels. The loss of predictive explanatory power is defined as the expectation of this predictive discrepancy. The goal is to find the simplest submodel which has a similar predictive distribution as the full model, that is, the simplest submodel whose loss of explanatory power is acceptable. To compute the expected predictive discrepancy between complex models, for which analytical solutions do not exist, we propose to use predictive distributions obtained via k-fold cross-validation. We compare the performance of the method to posterior probabilities (Bayes factors), deviance information criteria (DIC) and direct maximization of the expected utility via crossvalidation.