Abstract

If it is to qualify as knowledge, a learner's output should be accurate, stable and comprehensible. Learning multiple models can improve significantly on the accuracy and stability of single models, but at the cost of losing their comprehensibility (when they possess it, as do, for example, simple decision trees and rule sets). This article proposes and evaluates CMM, a meta-learner that seeks to retain most of the accuracy gains of multiple model approaches, while still producing a single comprehensible model. CMM is based on reapplying the base learner to recover the frontiers implicit in the multiple model ensemble. This is done by giving the base learner a new training set, composed of a large number of examples generated and classified according to the ensemble, plus the original examples. CMM is evaluated using C4.5RULES as the base learner, and bagging as the multiple-model methodology. On 26 benchmark datasets, CMM retains on average 60% of the accuracy gains obtained by baggin...

Citations

4071	C4.5: Programs for machine learning - Quinlan - 1993
3341	Pattern Classification and Scene Analysis - Duda, Hart - 1973
2529	C.J.E.: Uci repository of machine learning databases, http://www.ics.uci.edu/ ∼ mlearn/MLRepository.html - Blake, Keogh
1998	Bagging predictors - Breiman - 1996
1325	Experiments with a new boosting algorithm - Freund, Schapire - 1996
682	The CN2 induction algorithm - Clark, Niblett - 1989
582	Queries and concept learning - Angluin - 1988
547	P.: From data mining to knowledge discovery: An overview. In - Piatetsky-Shapiro, Fayyad, et al. - 1996
463	Stacked generalization - Wolpert - 1992
334	Improving generalization with active learning - Cohn, Atlas, et al. - 1994
251	Bagging, boosting, and C4.5 - Quinlan - 1996
176	Extracting refined rules from knowledge-based neural networks - Towell, Shavlik - 1993
166	On bias, variance, 0/1 - loss, and the curse-of-dimensionality, Data Mining and Knowledge Discovery 1 - Friedman - 1997
144	Bias plus variance decomposition for zero-one loss functions - Kohavi, Wolpert - 1996
131	Error-correcting output coding corrects bias and variance - Kong, Dietterich - 1995
129	Generating production rules from decision trees - Quinlan - 1987
97	A Comparative Evaluation of Voting and Meta-Learning of Partitioned Data - K, Stolfo - 1995
80	Megainduction: Machine Learning on Very large Databases - Catlett - 1991
77	A Theory of Learning Classification Rules - Buntine - 1992
65	Extracting Comprehensible Models from Trained Neural Networks - Craven - 1996
60	The effects of training set size on decision tree complexity - Oates, Jensen - 1997
29	Bayesian Model Averaging - Madigan, Raftery - 1996
22	Boosting and other machine learning algorithms - Drucker, Cortes, et al. - 1994
20	Learning prototypical concept descriptions - Datta, Kibler - 1995
11	Linear-time rule induction - Domingos - 1996
9	Comprehensible knowledge discovery in databases. In - Pazzani, Mani, et al. - 1997
5	Adjusting for multiple testing in decision tree pruning - Jensen - 1997
4	Weight averaging for neural networks and local resampling schemes. In: AAAI-96 Workshop on Integrating Multiple Learned Models for Improving and Scaling Machine Learning Algorithms - Utans - 1996
3	Process delay analysis using decision tree induction - Evans, Fisher - 1994
3	A distance metric for classification trees - Shannon - 1997
3	and the quantification of stability - Bias - 1995