We propose a general approach for estimating the parameters of latent variable probability models to maximize conditional likelihood and discriminant criteria. Unlike joint likelihood, these objectives are better suited for classification and regression. The approach utilizes and extends the previously introduced CEM framework (Conditional Expectation Maximization), which reformulates EM to handle the conditional likelihood case. We generalize the CEM algorithm to estimate any mixture of exponential family densities. This includes structured graphical models over exponential families, such as HMMs. The algorithm efficiently takes advantage of the factorization of the underlying graph. In addition, the new CEM bound is tighter and more rigorous than the original one. The final result is a CEM algorithm that mirrors the EM algorithm where both estimate a variational lower bound on their respective incomplete objective functions, and both generate the same standard M-steps ...
user correction - Legacy Corrections
In Advances in Neural Information Processing Systems 12