Abstract

Pruning is an effective method for dealing with noise in Machine Learning. Recently pruning algorithms, in particular Reduced Error Pruning , have also attracted interest in the field of Inductive Logic Programming . However, it has been shown that these methods can be very inefficient, because most of the time is wasted for generating clauses that explain noisy examples and subsequently pruning these clauses. We introduce a new method which searches for good theories in a top-down fashion to get a better starting point for the pruning algorithm. Experiments show that this approach can significantly lower the complexity of the task as well as increase predictive accuracy. OEFAI-TR-94-03 1 Introduction Pruning is a standard way of dealing with noise in Machine Learning. In particular in decision tree learning pruning methods proved to be most effective (see e.g. [Mingers, 1989] or [Esposito et al., 1993]). Pre-pruning --- heuristically deciding when to stop growing clauses and concep...

Citations

3143	Classification and Regression Trees - Breiman, Friedman, et al. - 1983
2888	Induction of Decision Trees - Quinlan - 1986
784	Learning logical definitions from relations - Quinlan - 1989
385	Very simple classification rules perform well on most commonly used datasets - Holte - 1993
156	An Empirical Comparison of Pruning Methods for Decision–Tree Induction - Mingers - 1989
116	Overfitting avoidance as bias - Schaffer - 1993
96	The effect of noise on concept learning - Quinlan - 1985
58	Handling noise in inductive logic programming - Dzeroski - 1991
57	Hydra: A Noise-Tolerant Relational Concept Learning Algorithm - Ali, Pazzani - 1993
50	An experimental comparison of human and machine learning formalisms - Muggleton, Bain, et al. - 1989
49	Information-based evaluation criterion for classifier's performance - Kononenko, Bratko - 1991
42	Efficient pruning methods for separate-and-conquer rule learning systems - Cohen - 1993
39	Compression, Significance and Accuracy - Muggleton, Srinivasan, et al. - 1992
24	Decision tree pruning as a search in the state space - Esposito, Malerba, et al. - 1993
23	FOSSIL: A Robust Relational Learner - Fürnkranz - 1994
12	Learning diagnostic rules from incomplete and noisy data - Bratko, Kononenko - 1987
2	Generality revisited - Flach - 1992
1	Generality revisited', in Logical Approaches to - Flach - 1992