Abstract

Variational inference methods, including mean field methods and loopy belief propagation, have been widely used for approximate probabilistic inference in graphical models. While often less accurate than MCMC, variational methods provide a fast deterministic approximation to marginal and conditional probabilities. Such approximations can be particularly useful in high dimensional problems where sampling methods are too slow to be effective. A limitation of current methods, however, is that they are restricted to parametric probabilistic models. MCMC does not have such a limitation; indeed, MCMC samplers have been developed for the Dirichlet process (DP), a nonparametric distribution on distributions (Ferguson, 1973) that is the cornerstone of Bayesian nonparametric statistics (Escobar & West, 1995; Neal, 2000). In this paper, we develop a meanfield variational approach to approximate inference for the Dirichlet process, where the approximate posterior is based on the truncated stick-breaking construction (Ishwaran & James, 2001). We compare our approach to DP samplers for Gaussian DP mixture models. 1.

Citations

787	Field Theory - Wit, Smith - 1986
717	Bayes factors - Kass, Raftery - 1995
656	An Introduction to Variational Methods for Graphical Models - Jordan, Ghahramani, et al. - 1999
488	T.S.: A Bayesian analysis of some nonparametric problems - Ferguson - 1973
375	Factorial hidden markov models - Ghahramani, Jordan - 1997
328	Hierarchical Dirichlet processes - Teh, Jordan, et al. - 2006
291	C.E.: Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems - Antoniak - 1974
285	Bayesian density estimation and inference using mixtures - Escobar, West - 1995
245	Markov chain sampling methods for Dirichlet process mixture models - Neal - 2000
181	Ferguson distributions via Pólya urn schemes - Blackwell, Macqueen - 1973
160	Gibbs sampling methods for stick–breaking priors - Ishwaran, James - 2001
127	Hierarchical topic models and the nested chinese restaurant process - Blei, Griffiths, et al. - 2003
89	Propagation algorithms for variational Bayesian learning - Ghahramani, Beal
53	Sampling theory of selectively neutral alleles - Ewens - 1972
25	SM: One long run with diagnostics: implementation strategies for Markov chain Monte Carlo. Stat Sci 1992, 7:493-497. Filho et al - AE, Lewis
24	Bayesian model search for mixture models based on optimizing variational bounds - Ueda, Ghahramani - 2002