Statistical applications in fields such as bioinformatics, information retrieval, speech processing, image processing and communications often involve large-scale models in which thousands or millions of random variables are linked in complex ways. Graphical models provide a general methodology for approaching these problems, and indeed many of the models developed by researchers in these applied fields are instances of the general graphical model formalism. We review some of the basic ideas underlying graphical models, including the algorithmic ideas that allow graphical models to be deployed in large-scale data analysis problems. We also present examples of graphical models in bioinformatics, error-control coding and language processing. Key words and phrases: Probabilistic graphical models, junction tree algorithm, sum-product algorithm, Markov chain Monte Carlo, variational inference, bioinformatics, error-control coding.

Abstract. Bayesian approaches to density estimation and clustering using mixture distributions allow the automatic determination of the number of components in the mixture. Previous treatments have focussed on mixtures having Gaussian components, but these are well known to be sensitive to outliers. This can lead to excessive sensitivity to small numbers of data points and consequent overestimates of the number of components. In this paper we develop a Bayesian approach to mixture modelling based on Student-Ø distributions, which are heavier tailed than Gaussians and hence more robust. By expressing the Student-Ø distribution as a marginalisation over additional latent variables we are able to derive a tractable variational inference algorithm for this model, which includes Gaussian mixtures as a special case. Results on a variety of real data sets demonstrate the improved robustness of our approach. 1

This is a synopsis of the work underlying the author’s contributed plenary presentation at the Valencia 8 meeting on Bayesian Statistics, held in Benidorm, June 2006. We show that Bayesian inference can be inconsistent under misspecification. Specifically, we exhibit a distribution P ∗ , a model M with P ∗ � ∈ M, and a prior Π on M such that the prior puts significant mass on ˜ P, the best approximation to P ∗ within the set M. Yet, if data are i.i.d. ∼ P ∗ , then for all large samples, the Bayesian posterior puts its mass on a subset of M that only contains bad approximations to P ∗. This result holds both if approximation quality is defined in terms of Kullback-Leibler divergence and if it is defined in terms of classification risk. We present several variations of this result, including one in which, with P ∗-probability 1, for all large enough samples, predictions of the next outcome based on the Bayesian predictive distribution become worse than predictions based on purely random guessing.

Abstract. The article contributes a derivation of variational Bayes for a large class of topic models by generalising from the well-known model of latent Dirichlet allocation. For an abstraction of these models as systems of interconnected mixtures, variational update equations are obtained, leading to inference algorithms for models that so far have used Gibbs sampling exclusively. 1

unstructured web log data using probabilistic topic modeling by

In a dynamic social or biological environment, the interactions between the underlying actors can undergo large and systematic changes. The latent roles or membership of the actors as determined by these dynamic links will also exhibit rich temporal phenomena, assuming a distinct role at one point while leaning more towards a second role at an another point. To capture this dynamic mixed membership in rewiring networks, we propose a state space mixed membership stochastic blockmodel which embeds an actor into a latent space and track its mixed membership in the latent space across time. We derived efficient approximate learning and inference algorithms for our model, and applied the learned models to analyze a social network between monks, and a rewiring gene interaction network of Drosophila melanogaster collected during its full life cycle. In both cases, our model reveals interesting patterns of the dynamic roles of the actors. 1. Introduction. Inference of network tomography