A gentle tutorial on the EM algorithm and its application to parameter estimation for gaussian mixture and hidden markov models

A gentle tutorial on the EM algorithm and its application to parameter estimation for gaussian mixture and hidden markov models (1997)

Cached

Download Links

by Jeff A. Bilmes

Citations:

BibTeX

@TECHREPORT{Bilmes97agentle,
    author = {Jeff A. Bilmes},
    title = {A gentle tutorial on the EM algorithm and its application to parameter estimation for gaussian mixture and hidden markov models},
    institution = {},
    year = {1997}
}

Years of Citing Articles

Bookmark

OpenURL

Abstract

We describe the maximum-likelihood parameter estimation problem and how the Expectation-form of the EM algorithm as it is often given in the literature. We then develop the EM parameter estimation procedure for two applications: 1) finding the parameters of a mixture of Gaussian densities, and 2) finding the parameters of a hidden Markov model (HMM) (i.e., the Baum-Welch algorithm) for both discrete and Gaussian mixture observation models. We derive the update equations in fairly explicit detail but we do not prove any convergence properties. We try to emphasize intuition rather than mathematical rigor. ii 1 Maximum-likelihood Recall the definition of the maximum-likelihood estimation problem. We have a density function ¢¡¤£¦ ¥ §© ¨ that is governed by the set of parameters § (e.g., might be a set of Gaussians and § could be the means and covariances). We also have a data set of size � , supposedly drawn from this distribution, i.e., �� £��£��© �. That is, we assume that these data vectors are independent and

Citations

6231	Maximum likelihood from incomplete data via the EM algorithm - Dempster, Laird - 1977
3909	Neural networks for pattern recognition - Bishop - 1995
1180	Fundamentals of speech recognition - Rabiner - 1993
634	Hierarchical mixtures of experts and the EM algorithm - Jordan, Jacobs - 1994
425	Mixture densities, maximum likelihood, and the EM algorithm - Redner, Walker - 1984
245	On the convergence properties of the EM algorithm - Wu - 1983
114	On convergence properties of the EM algorithm for Gaussian mixtures - Xu, Jordan - 1996
89	Convergence results for the em approach to mixtures of experts architectures - Jordan, Xu - 1995
49	Learning from incomplete data - Ghahramani, Jordan