Variational inference for Dirichlet process mixtures

Variational inference for Dirichlet process mixtures (2005)

Cached

Download Links

by David M. Blei , Michael I. Jordan

Venue:	Bayesian Analysis
Citations:	90 - 12 self

BibTeX

@ARTICLE{Blei05variationalinference,
    author = {David M. Blei and Michael I. Jordan},
    title = {Variational inference for Dirichlet process mixtures},
    journal = {Bayesian Analysis},
    year = {2005},
    volume = {1},
    pages = {121--144}
}

Years of Citing Articles

Bookmark

OpenURL

Abstract

Abstract. Dirichlet process (DP) mixture models are the cornerstone of nonparametric Bayesian statistics, and the development of Monte-Carlo Markov chain (MCMC) sampling methods for DP mixtures has enabled the application of nonparametric Bayesian methods to a variety of practical data analysis problems. However, MCMC sampling can be prohibitively slow, and it is important to explore alternatives. One class of alternatives is provided by variational methods, a class of deterministic algorithms that convert inference problems into optimization problems (Opper and Saad 2001; Wainwright and Jordan 2003). Thus far, variational methods have mainly been explored in the parametric setting, in particular within the formalism of the exponential family (Attias 2000; Ghahramani and Beal 2001; Blei et al. 2003). In this paper, we present a variational inference algorithm for DP mixtures. We present experiments that compare the algorithm to Gibbs sampling algorithms for DP mixtures of Gaussians and present an application to a large-scale image analysis problem.

Citations

3012	Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images - Geman, Geman - 1984
2333	Convex Analysis - Rockafellar - 1970
1370	Latent Dirichlet allocation - Blei, Ng, et al.
656	An introduction to variational methods for graphical models - Jordan, Ghahramani, et al. - 1999
588	Monte Carlo statistical methods - Robert, Casella - 2004
504	Nonlinear programming - Bertsekas - 1999
488	T.S.: A Bayesian analysis of some nonparametric problems - Ferguson - 1973
449	Probabilistic inference using Markov chain Monte Carlo methods - Neal - 1993
315	Sampling based approaches to calculating marginal densities - Gelfand, Smith - 1990
291	C.E.: Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems - Antoniak - 1974
285	Bayesian density estimation and inference using mixtures - Escobar, West - 1995
268	Graphical models, exponential families, and variational - Wainwright, Jordan - 2008
245	Markov chain sampling methods for Dirichlet process mixture models - Neal - 2000
238	Automatic image annotation and retrieval using cross-media relevance models - Jeon, Lavrenko, et al. - 2003
216	A constructive definition of Dirichlet priors - Sethuraman - 1994
181	Variational algorithms for approximate Bayesian inference - Beal - 2003
181	Ferguson distributions via Pólya urn schemes - Blackwell, Macqueen - 1973
160	Gibbs sampling methods for stick–breaking priors - Ishwaran, James - 2001
131	A variational Bayesian framework for graphical models - Attias - 2003
107	Fundamentals of Statistical Exponential Families: With Applications in Statistical Decision Theory - Brown - 1986
89	Propagation algorithms for variational Bayesian learning - Ghahramani, Beal
82	Estimating normal means with a conjugate style Dirichlet process prior - MacEachern - 1994
67	Monte Carlo statistical methods. Springer Texts in Statistics - Robert, Casella - 2004
33	editors. Advanced mean field methods: theory and practice - Opper, Saad - 2001
33	Variational approximations between mean field theory and the junction tree algorithm - Wiegerinck - 2000
29	VIBES: A variational inference engine for Bayesian networks - Bishop, Spiegelhalter, et al. - 2003
25	SM: One long run with diagnostics: implementation strategies for Markov chain Monte Carlo. Stat Sci 1992, 7:493-497. Filho et al - AE, Lewis
24	Concepts of independence for proportions with a generalization of the Dirichlet distribution - Connor, Mosimann - 1969
17	A computational approach for full nonparametric Bayesian inference under Dirichlet process mixture models - Gelfand, Kottas - 2002
11	Markov Chain Monte Carlo Methods in Practice - GILKS, RICHARDSON, et al. - 1996
1	Markov chain sampling methods for Dirichlet process mixture models - Blei, Jordan - 2000