Variational methods for model comparison have become popular in the neural computing/machine learning literature. In this paper we explore their application to the Bayesian analysis of mixtures of Gaussians. We also consider how the Deviance Information Criterion, or DIC, devised by Spiegelhalter et al. (2002), can be extended to these types of model by exploiting the use of variational approximations. We illustrate the results of using variational methods for model selection and the calculation of a DIC using real and simulated data. Using the variational approximation, one can simultaneously estimate component parameters and the model complexity. It turns out that, if one starts o# with a large number of components, superfluous components are eliminated as the method converges to a solution, thereby leading to an automatic choice of model complexity, the appropriateness of which is reflected in the DIC values.

Many databases involve ordered discrete responses in a temporal and spatial context, including, for example, land development intensity levels, vehicle ownership, and pavement conditions. An appreciation of such behaviors requires rigorous statistical methods, recognizing spatial effects and dynamic processes. This study develops a dynamic spatial ordered probit (DSOP) model in order to capture patterns of spatial and temporal autocorrelation in ordered categorical response data. This model is estimated in a Bayesian framework using Gibbs sampling and data augmentation, in order to generate all autocorrelated latent variables. It is found that the DSOP model yields much more accurate estimates than standard, non-spatial techniques. As for model selection, the DSOP model is clearly preferred to standard OP, dynamic OP and spatial OP models. These methods are then used to analyze land use changes over an 18-year period in Austin, Texas. In this analysis, temporal and spatial autocorrelation effects are found to be significantly positive. In addition, increases in travel times to the region’s central business district (CBD) are estimated to substantially reduce land development intensity. The proposed and tested DSOP model is felt to be a significant contribution to the field of spatial econometrics, where binary applications (for discrete response data) have been seen as the cutting edge. The Bayesian framework and Gibbs sampling techniques used here permit such complexity, in world of twodimensional autocorrelation.

The deviance information criterion (DIC) is widely used for Bayesian model comparison, despite the lack of a clear theoretical foundation. DIC is shown to be an approximation to a penalized loss function based on the deviance, with a penalty derived from a cross-validation argument. This approximation is valid only when the effective number of parameters in the model is much smaller than the number of independent observations. In disease mapping, a typical application of DIC, this assumption does not hold and DIC under-penalizes more complex models. Another deviance-based loss function, derived from the same decision-theoretic framework, is applied to mixture models, which have previously been considered an unsuitable application for DIC.

The evolution of land development in urban area has been of great interest to policy makers and planners. Due to the complexity of the land development process, no existing studies is considered sophisticated enough. This research uses Dynamic Spatial Ordered Probit (DSOP) model to analyze Austin’s land use intensity patterns over a 4-point panel. The observational units are 300m×300m grid cells derived from satellite images. The sample contains 2,771 such grid cells, spread among 57 zip code regions. The estimation suggests that increases in travel times to CBD substantially reduce land development intensity. More important, temporal and spatial autocorrelation effects are significantly positive, showing the superiority of the DSOP model. KEY WORDS: spatial autocorrelation, temporal dependency, ordered discrete response data, land development

We present recent approaches that extend standard Markov models and increase their modelling power. These capabilities are illustrated in the cited published works and more recently in the contributions to the Special Session on Markov models of the IEEE International Workshop on Machine Learning for Signal Processing, 2009. However, the review is not exhaustive and major older works may be missing. 1.

The variational approach to Bayesian inference enables simultaneous estimation of model parameters and model complexity. An interesting feature of this approach is that it appears also to lead to an automatic choice of model complexity. Empirical results from the analysis of hidden Markov models with Gaussian observation densities illustrate this. If the variational algorithm is initialised with a large number of hidden states, redundant states are eliminated as the method converges to a solution, thereby leading to an automatic selection of the number of hidden states. In addition, through the use of a variational approximation, the Deviance Information Criterion (DIC) for Bayesian model selection can be extended to the hidden Markov model framework. Calculation of the

This paper provides a brief review of the more popular methods for comparing models in a Bayesian framework. Personal experience in implementing these methods in problems requiring mixture models is also referenced. 1

Bayesian methods are widely used for selecting the number of components in a mixture models, in part because frequentist methods have difficulty in addressing this problem in general. Here we compare some of the Bayesianly motivated or justifiable methods for choosing the number of components in a one-dimensional Gaussian mixture model: posterior probabilities for a well-known proper prior, BIC, ICL, DIC and AIC. We also introduce a new explicit unit-information prior for mixture models, analogous to the prior to which BIC corresponds in regular statistical models. We base the comparison on a simulation study, designed to reflect published estimates of mixture model parameters from the scientific literature across a range of disciplines. We found that BIC clearly outperformed the five other

Using Bayesian inference to understand the allocation of resources between sexual and asexual reproduction

Abstract not found