Results 1 
5 of
5
The practical implementation of Bayesian model selection
 Institute of Mathematical Statistics
, 2001
"... In principle, the Bayesian approach to model selection is straightforward. Prior probability distributions are used to describe the uncertainty surrounding all unknowns. After observing the data, the posterior distribution provides a coherent post data summary of the remaining uncertainty which is r ..."
Abstract

Cited by 85 (3 self)
 Add to MetaCart
In principle, the Bayesian approach to model selection is straightforward. Prior probability distributions are used to describe the uncertainty surrounding all unknowns. After observing the data, the posterior distribution provides a coherent post data summary of the remaining uncertainty which is relevant for model selection. However, the practical implementation of this approach often requires carefully tailored priors and novel posterior calculation methods. In this article, we illustrate some of the fundamental practical issues that arise for two different model selection problems: the variable selection problem for the linear model and the CART model selection problem.
Bayesian Learning in Undirected Graphical Models: Approximate MCMC algorithms
, 2004
"... Bayesian learning in undirected graphical models  computing posterior distributions over parameters and predictive quantities  is exceptionally difficult. We conjecture that for general undirected models, there are no tractable MCMC (Markov Chain Monte Carlo) schemes giving the correct equilib ..."
Abstract

Cited by 36 (2 self)
 Add to MetaCart
Bayesian learning in undirected graphical models  computing posterior distributions over parameters and predictive quantities  is exceptionally difficult. We conjecture that for general undirected models, there are no tractable MCMC (Markov Chain Monte Carlo) schemes giving the correct equilibrium distribution over parameters. While this intractability, due to the partition function, is familiar to those performing parameter optimisation, Bayesian learning of posterior distributions over undirected model parameters has been unexplored and poses novel challenges. We propose several approximate MCMC schemes and test on fully observed binary models (Boltzmann machines) for a small coronary heart disease data set and larger artificial systems. While approximations must perform well on the model, their interaction with the sampling scheme is also important. Samplers based on variational meanfield approximations generally performed poorly, more advanced methods using loopy propagation, brief sampling and stochastic dynamics lead to acceptable parameter posteriors. Finally, we demonstrate these techniques on a Markov random field with hidden variables.
Bayesian Testing and Estimation of Association in a TwoWay Contingency Table
, 1996
"... In a twoway contingency table, one is interested in checking the goodness of fit of simple models such as independence, quasiindependence, symmetry, or constant association, and estimating parameters which describe the association structure of the table. In a large table, one may be interested in ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
In a twoway contingency table, one is interested in checking the goodness of fit of simple models such as independence, quasiindependence, symmetry, or constant association, and estimating parameters which describe the association structure of the table. In a large table, one may be interested in detecting a few outlying cells which deviate from the main association pattern in the table. Bayesian tests of the above hypotheses are described using a prior defined on the set of interaction terms of the loglinear model. These tests and associated estimation procedures have several advantages over classical fitting/estimation procedures First, the tests above can give measures of evidence in support of simple hypotheses. Second, the Bayes factors can be used to give estimates of association parameters of the table which allow for uncertainty that the hypothesized model is true. These methods are illustrated for a number of tables. Key words and phrases: Bayes factors, Laplace method, Gib...
392 MURRAY & GHAHRAMANI UAI 2004 Bayesian Learning in Undirected Graphical Models: Approximate MCMC algorithms
"... Bayesian learning in undirected graphical models—computing posterior distributions over parameters and predictive quantities— is exceptionally difficult. We conjecture that for general undirected models, there are no tractable MCMC (Markov Chain Monte Carlo) schemes giving the correct equilibrium di ..."
Abstract
 Add to MetaCart
Bayesian learning in undirected graphical models—computing posterior distributions over parameters and predictive quantities— is exceptionally difficult. We conjecture that for general undirected models, there are no tractable MCMC (Markov Chain Monte Carlo) schemes giving the correct equilibrium distribution over parameters. While this intractability, due to the partition function, is familiar to those performing parameter optimisation, Bayesian learning of posterior distributions over undirected model parameters has been unexplored and poses novel challenges. We propose several approximate MCMC schemes and test on fully observed binary models (Boltzmann machines) for a small coronary heart disease data set and larger artificial systems. While approximations must perform well on the model, their interaction with the sampling scheme is also important. Samplers based on variational meanfield approximations generally performed poorly, more advanced methods using loopy propagation, brief sampling and stochastic dynamics lead to acceptable parameter posteriors. Finally, we demonstrate these techniques on a Markov random field with hidden variables. 1
SPARSITY MODELING FOR HIGH DIMENSIONAL SYSTEMS: APPLICATIONS IN GENOMICS AND STRUCTURAL BIOLOGY
"... The availability of very high dimensional data has brought sparsity modeling to the forefront of statistical research in recent years. From complex physical models with hundreds of parameters to DNA microarrays which offer observations in tens to hundreds of thousands of dimensions, separating relev ..."
Abstract
 Add to MetaCart
The availability of very high dimensional data has brought sparsity modeling to the forefront of statistical research in recent years. From complex physical models with hundreds of parameters to DNA microarrays which offer observations in tens to hundreds of thousands of dimensions, separating relevant and irrelevant parameters is becoming more and more important. This dissertation will focus on innovations in the area of variable and model selection as they pertain to these high dimensional systems. Chapter 1 will discuss work from the literature on the areas of variable and model selection. Chapter 2 will describe an innovation to hierarchical variable selection modeling that corrects errors that stem from assuming incorrectly that multiple thousands of observations are informing about the same distribution. In Chapter 3, we introduce a novel technique for applying variable selection priors to induce sparsity in variance modeling.