Abstract

One of the basic tasks for Bayesian networks (BNs) is that of learning a network structure from data. The BN-learning problem is NPhard, so the standard solution is heuristic search. Many approaches have been proposed for this task, but only a very small number outperform the baseline of greedy hill-climbing with tabu lists; moreover, many of the proposed algorithms are quite complex and hard to implement. In this paper, we propose a very simple and easy-toimplement method for addressing this task. Our approach is based on the well-known fact that the best network (of bounded in-degree) consistent with a given node ordering can be found very efficiently. We therefore propose a search not over the space of structures, but over the space of orderings, selecting for each ordering the best network consistent with it. This search space is much smaller, makes more global search steps, has a lower branching factor, and avoids costly acyclicity checks. We present results for this algorithm on both synthetic and real data sets, evaluating both the score of the network found and in the running time. We show that orderingbased search outperforms the standard baseline, and is competitive with recent algorithms that are much harder to implement. 1

Citations

5662	Probabilistic reasoning in intelligent systems: networks of plausible inference - Pearl - 1988
1583	Estimating the dimension of a model - Schwarz - 1978
876	A Bayesian method for the induction of probabilistic networks from data - Cooper, Herskovits - 1992
751	Learning Bayesian networks: the combination of knowledge and statistical data - Heckerman, Geiger, et al. - 1995
245	Genomic expression programs in the response of yeast cells to environmental changes - Gasch, Spellman, et al. - 2000
227	Functional discovery via a compendium of expression profiles - Hughes, Marton, et al. - 2000
201	The ALARM monitoring system: A case study with two probabilistic inference techniques for belief networks - Beinlich, Suermondt, et al. - 1989
156	Being Bayesian about network structure. a Bayesian approach to structure discovery in Bayesian networks - Friedman, Koller
153	Theory refinement on bayesian networks - Buntine - 1991
141	Learning Bayesian network structure from massive datasets: The ”sparse candidate” algorithm - Friedman, Nachman, et al. - 1999
120	Learning Bayesian networks is NP-complete - Chickering - 1996
110	Learning Equivalence Classes of Bayesian Network Structure - Chickering - 1996
108	Cached Sufficient Statistics for Efficient Machine Learning with Large Datasets - Moore, Lee - 1998
107	Learning belief networks in the presence of missing values and hidden variables - Friedman - 1997
98	D.: Learning bayesian networks is np-hard - Chickering, Geiger, et al. - 1994
45	Learning bayesian network structures by searching for the best ordering with genetic algorithms - Larrañaga, Kuijpers, et al. - 1996
33	Optimal reinsertion: A new search operator for accelerated and more accurate bayesian network structure learning - Moore, Wong - 2003
29	Data perturbation for escaping local maxima in learning - Elidan, Ninio, et al. - 2002
27	A model-based approach to insulin adjustment - Andreassen, Hovorka, et al. - 1991
5	On the use of skeletons when learning in Bayesian networks - Steck - 2000