Abstract

In recent years there has been a flurry of works on learning Bayesian networks from data. One of the hard problems in this area is how to effectively learn the structure of a belief network from incomplete data---that is, in the presence of missing values or hidden variables. In a recent paper, I introduced an algorithm called Structural EM that combines the standard Expectation Maximization (EM) algorithm, which optimizes parameters, with structure search for model selection. That algorithm learns networks based on penalized likelihood scores, which include the BIC/MDL score and various approximations to the Bayesian score. In this paper, I extend Structural EM to deal directly with Bayesian model selection. I prove the convergence of the resulting algorithm and show how to apply it for learning a large class of probabilistic models, including Bayesian networks and some variants thereof.

Citations

6232	Maximum likelihood from incomplete data via the EM algorithm - Dempster, Laird, et al. - 1977
877	A Bayesian method for the induction of probabilistic networks from data - Cooper, Herskovits - 1992
416	Bayesian classification (AutoClass): Theory and results - Cheeseman, Stutz - 1996
248	A tutorial on learning Bayesian networks - Heckerman - 1995
240	Context-specific independence in bayesian networks - Boutilier, Friedman, et al. - 1996
208	Learning Bayesian networks with local structure - Friedman, Goldszmidt - 1999
179	The EM algorithm for graphical association models with missing data - Lauritzen - 1995
170	D: Inference and missing data - Rubin - 1976
164	F: Learning Bayesian belief networks; An approach based on the MDL principle. Comput Intell - Lam, Bacchus - 1994
155	Efficient approximations for the marginal likelihood of Bayesian networks with hidden variables - Chickering, Heckerman - 1997
152	A Bayesian approach to learning Bayesian networks with local structure - Chickering, Heckerman, et al. - 1997
133	Adaptive probabilistic networks with hidden variables - Binder, Koller, et al. - 1997
112	Learning classification trees - BUNTINE - 1992
102	Mean field theory for sigmoid belief networks - Zemel, Saul, et al. - 1996
81	Knowledge representation and inference in similarity networks and bayesian multinets - Geiger, Heckerman - 1996
25	Estimating dependency structure as a hidden variable - Meila, Jordan - 1998
22	The alarm monitoring system - Beinlich - 1989
20	Learning Bayesian networks in the presence of missing values and hidden variables - Friedman - 1997
16	Learning mixtures of Bayesian networks - Thiesson, Meek, et al. - 1998
13	Learning Bayesian networks from incomplete data - Singh - 1997
9	1996], ‘Ensemble learning for hidden Markov models’. URL: ftp://wol.ra.phy.cam.ac.uk/pub/mackay/ensemblePaper.ps.gz - Macphee, MacKay
4	Asymptotic model selection for directed graphs with hidden variables - Geiger, Heckerman, et al. - 1996
1	Learning Bayesian networks: Thecombinationofknowledgeand statistical data - Heckerman, Geiger, et al. - 1995
1	Learning Bayesian networks: The combinationof knowledgeand statistical data - Heckerman, Geiger, et al. - 1995