Abstract

This paper is a survey of inductive rule learning algorithms that use a separate-and-conquer strategy. This strategy can be traced back to the AQ learning system and still enjoys popularity as can be seen from its frequent use in inductive logic programming systems. We will put this wide variety of algorithms into a single framework and analyze them along three different dimensions, namely their search, language and overfitting avoidance biases.

Citations

6071	Genetic Algorithms - Goldberg - 1989
4071	C4.5: Programs for machine learning - Quinlan - 1993
3143	Classification and Regression Trees - Breiman, Friedman, et al. - 1984
2888	Induction of Decision Trees - Quinlan - 1986
2801	Optimization by simulated annealing - Kirkpatrick, Gelatt, et al. - 1983
962	Efficient induction of logic programs - Muggleton, Feng - 1990
892	Modeling by shortest data description - Rissanen - 1978
800	Fast effective rule induction - Cohen - 1995
784	Learning logical definitions from relations - Quinlan - 1989
728	A Formal Basis for the Heuristic Determination of Minimum Cost Paths - Hart, Nilsson, et al. - 1968
682	The CN2 induction algorithm - Clark, Niblett - 1989
637	A theory and methodology of inductive learning - Michalski - 1983
560	Inverse entailment and progol - Muggleton - 1995
517	Generalization as search - Mitchell - 1982
347	Learning decision lists - Rivest - 1987
296	The multi-purpose incremental learning system aq15 and its testing application to three medical domains - Michalski, Mozetic, et al. - 1986
292	Learning Efficient Classification Procedures and their Application to Chess End Games - Quinlan - 1983
291	Rule induction with cn2: Some recent improvements - Clark, Boswell - 1991
270	An information measure for classification - Wallace, Boulton - 1968
190	Boolean feature discovery in empirical learning - Pagallo, Haussler - 1990
167	The Need for Biases in Learning Generalizations - Mitchell - 1990
147	An Empirical Comparison of Selection Measures for Decision–Tree Induction - Mingers - 1989
140	The utility of knowledge in inductive learning - Pazzani, Kibler - 1992
136	Concept Learning and the Problem of Small Disjuncts - Holte, Acker, et al. - 1989
129	Generating production rules from decision trees - Quinlan - 1987
116	Overfitting avoidance as bias - Schaffer - 1993
113	Shift of bias for inductive concept learning - Utgoff - 1986
101	Incremental Reduced Error Pruning - Fürnkranz, Widmer - 1994
95	Controlling the complexity of learning in logic through syntactic and task-oriented models - Kietz, Wrobel - 1992
94	Wrappers for performance enhancement and oblivious decision graphs - Kohavi
91	A Further Comparison of Splitting Rules for Decision-Tree Induction - Buntine, Niblett - 1992
91	PRISM: an algorithm for inducing modular rules - Cendrowska - 1987
80	Inductive constraint logic - Raedt, Laer - 1995
77	Unifying instance–based and rule–based induction - Domingos - 1996
76	Cameron-Jones. Oversearching and Layered Search in Empirical Learning - Quinlan, M - 1995
71	Interactive Theory Revision: An Inductive Logic Programming Approach - Raedt - 1992
70	OPUS: An Efficient Admissible Algorithm for Unordered Search - Webb - 1995
68	Induction of firstorder decision lists: results on learning the past tense of English verbs - Mooney, Califf - 1995
63	Learning nonrecursive definitions of relations with LINUS - Lavrac, Dzeroski, et al. - 1991
62	A note on inductive generalisation - Plotkin - 1970
61	Grammatically biased learning: Learning logic programs using an explicit antecedent description language - Cohen - 1994
60	Structural regression trees - Kramer - 1996
58	Handling noise in inductive logic programming - Dzeroski - 1991
57	Hydra: A Noise-Tolerant Relational Concept Learning Algorithm - Ali, Pazzani - 1993
57	The necessary and sufficient conditions for the uniform convergence of averages to their expected values. Teoriya Veroyatnostei i Ee Primeneniya - Vapnik, Chervonenkis - 1981
56	Determinate literals in inductive logic programming - Quinlan - 1991
55	Non-monotonic learning - Bain, Muggleton - 1992
54	Induction of logic programs: FOIL and related systems - Quinlan, Cameron-Jones - 1995
53	Inductive Logic Programming: from Machine Learning to Software Engineering - Bergadano, Gunetti - 1995
51	Relational Clichs: Constraining Constructive Induction During Relational learning - Silverstein, Pazzani - 1991