This paper is a survey of inductive rule learning algorithms that use a separate-and-conquer strategy. This strategy can be traced back to the AQ learning system and still enjoys popularity as can be seen from its frequent use in inductive logic programming systems. We will put this wide variety of algorithms into a single framework and analyze them along three different dimensions, namely their search, language and overfitting avoidance biases.

Abstract. This paper presents a comparison between two feature selection methods, the Importance Score (IS) which is based on a greedy-like search and a genetic algorithm-based (GA) method, in order to better understand their strengths and limitations and their area of application. The results of our experiments show a very strong relation between the nature of the data and the behavior of both systems. The Importance Score method is more efficient when dealing with little noise and small number of interacting features, while the genetic algorithms can provide a more robust solution at the expense of increased computational effort.

This paper reports on three studies comparing symbolic and subsymbolic methods for concept learning from examples. The first study compared five learning methods, three representing symbolic learning paradigm---ecision tree learning (C4.5), role learning (AQ15), and constructive rule learning (AQ17-HCI), and the other two representing the subsymbolic paradigm--- neural net learning using backpropagation (BpNet), and a classifier system employing genetic algorithm (CFS). All methods have been experimentally applied to learn several different DNF-type concepts (i.e. concepts representable by a simple DNF expression). The second study compared performance of a large number of learning programs on learning DNF-type concepts from data with and without noise, and a non-DNF-type "m-of-n" concept. The third study compared genetic algorithm based learning (GABIL and Adaptive GABIL) with decision tree learning (C4.5) and decision rule learning (AQ14) on twelve DNF-type concepts. In all studies, symbolic methods, in particular those applying constructive induction, outperformed subsymbolic methods in learning DNF-type concepts from data both without and with noise. In case of learning non-DNF-type concepts, symbolic methods without constructive induction performed worse, but those with constructive induction matched the performance of neural network methods.

This paper presents a method for constructive induction in which new problem-relevant atu'ibutes are generated by analyzing iteratively created inductive hypotheses. The method starts by creating a set of rules from given examples using the AQ algorithm. These rules are then evaluated according to a "rule quality criterion." Subsets of the best-performing rules for each decision class are selected to form new attributes. These new attributes are used to reformulate the training examples used in the previous step, and the whole inductive process starts again. This iterative process ends when the performance of the rules exceeds a determined threshold. In the experiments on learning different DNF functions, the method outperformed in terms of predictive accuracy both, the AQ15 role learning method, as well as the REEDWOOD decision tree learning method.

This paper addresses the problem of learning evolving concepts, that is, concepts whose meaning gradually evolves in time. Solving this problem is important to many applications, for example, building intelligent agents for helping users in Internet search, active vision, automatically updating knowledge-bases, or acquiring profiles of users of telecommunication networks. Requirements for a learning architecture supporting such applications include the ability to incrementally modify concept definitions to accommodate new information, fast learning and recognition rates, low memory needs, and the understandability of computer-created concept descriptions. To address these requirements, we propose a learning architecture based on Variable-Valued Logic, the Star Methodology, and the AQ algorithm. The method uses a partial-memory approach, which means that in each step of learning, the system remembers the current concept descriptions and specially selected representative examples from th...

Abstract not found

The paper discusses and experimentally compares five different methods for concept learning from examples. The first three are symbolic methods, specifically, a decision tree learning method (C4.5), a rule learning method (AQ15), and a constructive rule learning method (AQ17-HCI). The other two are nonsymbolic methods, one, a neural net trained by a backpropagation algorithm, (BpNet), and a second, classifier system employing a genetic algorithm (CFS). All methods have been experimentally applied to various concept learning problems. This paper reports the first phase of experiments where concepts to be learned were proposed by human subjects, and thus "cognitively oriented." The second phase will involve learning other types concepts. To analyze the performance of the programs, a diagrammatic visualization system, DIAV, was employed. DIAV presents learned and target concepts as images in a planar model of a multidimensional space, and permits one to visualize exact error of a learning process. In several experiments, symbolic methods, in particular the AQ17-HCI method, consistently outperformed subsymbolic methods in terms of both, predictive accuracy and simplicity of learned descriptions.

Abstract not found

In this paper, we describe a method for learning shape descriptions of objects in x-ray images. The descriptions are induced from shape examples using the AQ15c inductive learning system. The method has been experimentally compared to k-nearest neighbor, a statistical pattern recognition technique, the C4.5 decision tree learning program, and a multilayer feed-forward neural network. Experimental results demonstrate strong advantages of the AQ methodology over the other methods. Specifically, the method has higher predictive accuracy and faster learning and recognition rates. AQ's representation language, VL 1 , was better suited for this problem, which can be seen by examining the empirical results and the learned rules. The method was applied to the problem of detecting blasting caps in x-ray images of luggage. An intelligent system performing this detection task can be used to assist airport security personnel with luggage screening. 1. INTRODUCTION Despite many efforts, the proble...

The paper describes a novel method for consreactive induction, called PRAX (Principal Axes). The madeflying idea of the method is to determine descriptions of a class of certain basic concepts, and then use these descriptions as "principal axes" with which all other concepts can be described. Given examples of a new concept, the system determines a similarity matrix (SM) for that concept, that contains the average degree of similarity between the concept examples and the principal axes. These degrees of similarity are viewed as newly constructed attributes. To recognize an unknown concept instance, the method creates an SM for it, and then seeks the best matching similarity realfix of all known concepts. In experimental testing of the method on the problem of learning descriptions of a large number of visual textttres, the PRAX method significantly outperformed the k-NN classifier often used for such problems. A very important result of this research is a demonstration that a symbolic learning method can be successfully applied to the domain of continuous attributes of low level vision in which nonsymbolic methods have been waditionally employed.