In principle, the Bayesian approach to model selection is straightforward. Prior probability distributions are used to describe the uncertainty surrounding all unknowns. After observing the data, the posterior distribution provides a coherent post data summary of the remaining uncertainty which is relevant for model selection. However, the practical implementation of this approach often requires carefully tailored priors and novel posterior calculation methods. In this article, we illustrate some of the fundamental practical issues that arise for two different model selection problems: the variable selection problem for the linear model and the CART model selection problem.

We propose a hierachical full Bayesian model for radial basis networks. This model treats the model dimension (number of neurons), model parameters,...

this paper is to provide a Bayesian algorithm which mimics the MARS procedure. This is done by considering the number of basis functions, along with their type (see Section 2.1), their coefficients and their form (the positions of the split points and the sign indicators) random. We treat these as additional parameters in the problem and make inference about them using the data. The problem of routine calculation of the posterior distribution of the models is addressed by designing a suitable Markov chain Monte Carlo (MCMC) reversible jump simulation algorithm as set out by Green (1995). The simulated sample contains many different MARS models with corresponding posterior weights but if a estimate for f with high predictive power is all that is required then pointwise averaging over all the models in the sample is suggested. This work is an extension to the Bayesian approach to curve fitting in one dimension given by Denison et al. (1998b) and is related to the Bayesian CART algorithms proposed by Denison et al. (1998a) and Chipman et al.

We present a Bayesian analysis of a piecewise linear model constructed using basis functions which generalises the univariate linear spline to higher dimensions. Prior distributions are adopted on both the number and locations of the splines which leads to a model averaging approach to prediction with predictive distributions that take into account model uncertainty. Conditioning on the data produces a Bayes local linear model with distributions on both predictions and on local linear parameters. The method is spatially adaptive and covariate selection is achieved by using splines of lower dimension than the data. KEYWORDS: Bayesian piecewise linear regression; Bayesian model averaging; nonlinear regression; multivariate splines; local linear regression. 1 1 Introduction Many methods exist for modelling the mean regression surface, E(Y jX), of a response variable Y , given a design matrix of covariates or predictors X 2 R p . Each method has associated benets and drawba...

In this paper, we propose to construct the networks according to the following stages. Firstly, we determine the number of possible parent gene sets and the input sets of gene variables corresponding to each gene, and this is done by using a novel clustering technique based on mutual information minimization. Simulated annealing is employed to solve the optimization problem. After such initial gene clustering, we restrict our attention to the class of different functions from the possible parent gene sets to each target gene. Secondly, each function is then modelled by a perceptron consisting of a linear term and a nonlinear term. A reversible jump Markov chain Monte Carlo (MCMC) technique is used to calculate the model order and the parameters. Finally, coefficient of determination (CoD) is employed to compute the probability of selecting different predictors for each gene. To test this approach for constructing gene regulatory networks, we have carried out computational experiments using data from known gene response pathways including ionizing radiation and downstream targets of inactivating gene mutations.

Bayesian methods are developed for the seemingly unrelated regressions (SUR) model where the model order or structure is presumed random. In particular we consider the class of models that are linear in some basis space. This class includes standard linear regression as a special case, as well as those models that involve nonlinear transformations of the explanatory variables through a set of basis functions. Applications are given for vector autoregression (VAR) models of unknown order and multivariate splines with unknown knot points. Keywords: Multiple response regression

A common problem in statistics, and other disciplines, is to approximate adequately a function of several variables. In this paper we review some possible nonparametric Bayesian models with which we can perform this multiple regression problem. We shall also demonstrate how these basic models can be extended to allow the analysis of time series, both conventional and financial, survival analysis and space-time data. This paper is a brief review of some of the work that appeared in Denison (1997) together with some new research that has since taken place, following on from the themes of this thesis. Keywords: Markov chain Monte Carlo; multiple regression; multivariate adaptive regression splines. 1 Introduction 1.1 Multiple regression A great deal of statistics involves regression analysis. Approximating the relationship between a response (or dependent) variable and a set of predictor (or independent) variables is of fundamental importance in understanding the underlying structure i...

In this article we demonstrate how to generate independent and identically distributed samples from the model space of the Bayes linear model with orthogonal predictors. We use the method of coupled Markov chains from the past as introduced by Propp and Wilson (1996). This procedure alleviates any concerns over convergence and sample mixing. We present a number of examples including a perfect simulation of Bayesian wavelet selection in a 1024 dimensional model space, a knot selection problem for spline smoothers and, a standard linear regression variable selection problem. Keywords: Exact sampling, perfect simulation, wavelets, variable selection, Markov chain Monte Carlo, knot selection, radial basis functions. 1 Introduction Accounting for model uncertainty is an important issue in statistical data modelling. Failure to do so can lead to poorer performance and over confident predictions (Draper, 1995). An important component of model uncertainty is determining which predictors to ...

Wavelet Networks (WNs) were introduced in 1992 as a combination of... In this paper, we analyze some of their properties and hightlight their advantages for object representation purposes. We then present a series of experimental results where we have used WNs for face tracking in which we exploit the efficiency due to data reduction, for face recognition and face-pose estimation where we exploit the optimized filter bank principle of the WNs.

Generalized linear models are one of the most widely used tools of the data analyst. However, the model assumes that the structure of the regression relationship between the response and the covariates is linear on a known transformed scale. We focus here on different methods to perform the same type of analyses. These involve using nonparametric models to determine the relationship between the response and covariates after the usual transformation has been carried out. We demonstrate how such a semiparametric model performs for binary regression. 1 Introduction Regression techniques are among some of the most widely used methods in applied statistics. Given a response variable Y , and a set of covariates X = (X 1 ; X 2 ; \Delta \Delta \Delta ; X p ), one is often interested in estimating an assumed functional relationship between Y and X, and in predicting further responses for new values of the covariates. One way of modelling such a relationship is to present the expected value of...