Abstract

Recent work in supervised learning has shown that a surprisingly simple Bayesian classifier with strong assumptions of independence among features, called naive Bayes, is competitive with state of the art classifiers such as C4.5. This fact raises the question of whether a classifier with less restrictive assumptions can perform even better. In this paper we examine and evaluate approaches for inducing classifiers from data, based on recent results in the theory of learning Bayesian networks. Bayesian networks are factored representations of probability distributions that generalize the naive Bayes classifier and explicitly represent statements about independence. Among these approaches we single out a method we call Tree Augmented Naive Bayes (TAN), which outperforms naive Bayes, yet at the same time maintains the computational simplicity (no search involved) and robustness which are characteristic of naive Bayes. We experimentally tested these approaches using benchmark problems from the U. C. Irvine repository, and compared them against C4.5, naive Bayes, and wrapper-based feature selection methods. 1

Citations

7321	Introduction to Algorithms - Cormen, Leiserson, et al. - 2001
5664	Probabilistic Reasoning in Intelligent Systems - Pearl - 1988
4071	C4.5: Programs for machine learning - Quinlan - 1993
752	Learning Bayesian networks: the combination of knowledge and statistical data - Heckerman, Geiger, et al. - 1995
662	Repository of machine learning databases - Murphy, Aha - 1994
531	Multi-interval discretization of continuousvalued attributes for classification learning - Fayyad, Irani - 1993
528	A study of cross-validation and bootstrap for accuracy estimation and model selection - Kohavi - 1995
515	K: Irrelevant features and the subset selection problem - GH, Kohavi, et al.
487	Approximating discrete probability distributions with dependence trees - Chow, Liu - 1968
335	Supervised and unsupervised discretization of continuous features - Dougherty, Kohavi, et al. - 1995
285	An analysis of Bayesian classifiers - Langley, Iba, et al. - 1992
248	A tutorial on learning Bayesian networks - Heckerman - 1995
179	Induction of selective bayesian classifiers - Langley, Sage - 1994
164	F: Learning Bayesian belief networks; An approach based on the MDL principle. Comput Intell - Lam, Bacchus - 1994
91	MLC++: A machine learning library in C - Kohavi, John, et al. - 1994
58	Searching for dependencies in Bayesian classifiers - Pazzani - 1995
50	Discretization of continuous attributes while learning Bayesian networks - Friedman, Goldszmidt - 1996
28	An entropy-based learning algorithm of Bayesian conditional trees - Geiger
1	A study of cross-validationand bootstrap for accuracy estimation and model selection - Kohavi - 1995