Abstract Despite the fact that many symbolic and neural network (connectionist) learning algorithms address the same problem of learning from classified examples, very little is known regarding their comparative strengths and weaknesses. Experiments comparing the ID3 symbolic learning algorithm with the perception and backpropagation neural learning algorithms have been performed using five large, real-world data sets. Overall, backpropagation performs slightly better than the other two algorithms in terms of classification accuracy on new examples, but takes much longer to train. Experimental results suggest that backpropagation can work significantly better on data sets containing numerical data. Also analyzed empirically are the effects of (1) the amount of training data, (2) imperfect training examples, and (3) the encoding of the desired outputs. Backpropagation occasionally outperforms the other two systems when given relatively small amounts of training data. It is slightly more accurate than ID3 when examples are noisy or incompletely specified. Finally, backpropagation more effectively utilizes a "distributed " output encoding.

Despite the fact that many symbolic and connectionist (neural net) learning algorithms are addressing the same problem of learning from classified examples, very little Is known regarding their comparative strengths and weaknesses. This paper presents the results of experiments comparing the ID3 symbolic learning algorithm with the perceptron and back-propagation connectionist learning algorithms on several large real-world data sets. The results show that ID3 and perceptron run significantly faster than does backpropagation, both during learning and during classification of novel examples. However, the probability of correctly classifying new examples is about the same for the three systems. On noisy data sets there is some indication that backpropagation classifies more accurately. 1.

Abstract. Supervised learning in attribute-based spaces is one of the most popular machine learning problems studied and, consequently, has attracted considerable attention of the genetic algorithm community. The fullmemory approach developed here uses the same nigh-level descriptive language that is used in rule-based systems. This allows for an easy utilization of inference rules of the well-known inductive learning methodology, which replace the traditional domain-independent operators and make the search task-specific. Moreover, a closer relationship between the underlying task and the processing mechanisms provides a setting for an application of more powerful task-specific heuristics. Initial results obtained with a prototype implementation for the simplest case of single concepts indicate that genetic algorithms can be effectively used to process nigh-level concepts and incorporate task-specific knowledge. The method of abstracting the genetic algorithm to the problem level, described here for the supervised inductive learning, can be also extended to other domains and tasks, since it provides a framework for combining recently popular genetic algorithm methods with traditional problem-solving methodologies. Moreover, in this particular case, it provides a very powerful tool enabling study of the widely accepted but not so well understood inductive learning methodology.

Genetic Algorithms (GAs) have traditionally been used for non-symbolic learning tasks. In this paper we consider me application of a GA to a symbolic learning task, supervised concept learning from examples. A GA concept learner (GABL) is implemented ahat learns a concept from a set of positive and negative examples. GABL is run in a batchincremental mode to facilitate comparison with an incremental concept learner, IDSR. Preliminary results suppon ahat. despite minimal system bias, GABL is an ' effective concept learner and is quite competitive with IDSR as me target concept increases in complexity. 1.

The paper discusses and experimentally compares five different methods for concept learning from examples. The first three are symbolic methods, specifically, a decision tree learning method (C4.5), a rule learning method (AQ15), and a constructive rule learning method (AQ17-HCI). The other two are nonsymbolic methods, one, a neural net trained by a backpropagation algorithm, (BpNet), and a second, classifier system employing a genetic algorithm (CFS). All methods have been experimentally applied to various concept learning problems. This paper reports the first phase of experiments where concepts to be learned were proposed by human subjects, and thus "cognitively oriented." The second phase will involve learning other types concepts. To analyze the performance of the programs, a diagrammatic visualization system, DIAV, was employed. DIAV presents learned and target concepts as images in a planar model of a multidimensional space, and permits one to visualize exact error of a learning process. In several experiments, symbolic methods, in particular the AQ17-HCI method, consistently outperformed subsymbolic methods in terms of both, predictive accuracy and simplicity of learned descriptions.

10 The Author 13 Acknowledgements 15 1 Introduction 16 1.1 Definition of Learning : : : : : : : : : : : : : : : : : : : : : : : : 16 1.2 The objectives of ML : : : : : : : : : : : : : : : : : : : : : : : : : 17 1.3 Approaches taken so far : : : : : : : : : : : : : : : : : : : : : : : 18 1.4 Motivation for the project : : : : : : : : : : : : : : : : : : : : : : 20 1.5 The Structure of the Thesis : : : : : : : : : : : : : : : : : : : : : 21 2 Theory of Inductive Learning 22 2.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 22 2.2 Induction as a Search : : : : : : : : : : : : : : : : : : : : : : : : : 23 2.2.1 The Goal: Hypothesis : : : : : : : : : : : : : : : : : : : : 24 2.2.2 The Search Space: Hypothesis Space : : : : : : : : : : : : 24 2.2.3 The operators : : : : : : : : : : : : : : : : : : : : : : : : : 26 2.3 Approaches : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 27 2.3.1 Statistical Classification : : : : : : : : : : : : : : : : : : : 27...

The behaviour of a learning program as the quantity of data is increased affects to a large extent its applicability on real-world problems. This paper presents the results of a theoretical and experimental investigation of the scalability of four well-known empirical concept-learning programs. In particular it examines the effect of using numeric features in the training set. The theoretical part of the work involved a detailed worstcase computational complexity analysis of the algorithms. The results of the analysis deviate substantially from previously reported estimates, which have mainly examined discrete and finite feature spaces. In order to test these results, a set of experiments was carried out, involving one artificial and two real data sets. The artificial data set introduces a near-worst-case situation for the examined algorithms, while the two real data sets provide an indication of their average-case behaviour. Keywords: empirical concept learning, scalability, decision ...

algorithms, it Ires hecn shown that the ID3 s),mbolic learning algorithm l_eC[)rms better than a neural mrtwork trained using the backpropagation learning rule. llowever, none of the previous studies rompared the pelfi)rmance of the two lea.rning approaches for distortion invariant object recognmon. Within this domain, we compared the ID3 system and a higher-order neural network (tlONN). Our results sl,ow that tfONNs are superior to ID3 with respect to recognition accuracy whereas, on a sequential machine, ID3 classifies examples]'aster once trained. A further advantage of tlONNs is the small training set required, tlONNs can be trained on just one view of each object, whereas ID3 needs an exhaustive training set.

Four different learning methods are experimentally compared by applying them to a series of simple, single concept learning problems. The methods compared include a rule-learning program, AQI$, a decision tree learning program, C4.$ (a successor of ID3), a program simulating a neural net trained by a backpropagation algorithm, BpNet, and a classifier system based on a genetic algorithm, CFS. The comparison employs a novel diagrammatic visualization technique that graphically represents training examples, the target and learned concepts, as well as the exact error image. The study described here is concerned with learning concepts generated by human subjects, thus, it is biased toward learning human-style descriptions (the next phase of research will consider other types of concepts). The results of experiments were that the ranking of the methods (from the lowest to the highest) on the basis of the average accuracy of the descriptions learned was the same as ranking on the basis of the simplicity of the descriptions, namely, the CFS method, BpNet, C4.$ and AQ15.

This paper justifies and demonstrates a machine learning approach to the problem of texture recognition. The learning-based texture recognition is separated into the following phases: (i) the acquisition of texture concepts, (ii) the optimization of concept prototypes, and (iii) the recognition of unknown texture samples. Methodology adapted to the acquisition and recognition of-noisy texture data are introduced based on the AQ learning-from-examples algorithm. Characteristics of learning-based recognition of texture concepts are presented for different parameters of attribute extraction, different number of training data, and for different setting of the learning tooL Special emphasis is given to the optimization of noisy te.xture concepts. The optimization model and processes are designed to improve system recognition effectiveness according to given optimization criteria and evaluation measures. These criteria and measures are designed regarding the texture recognition and segmentation tasks. Various concept optimization methods are presented and tested. The empirical evaluation of developed learning-based approach to texture recognition is-demonstrated on the domain composed of twelve texture classes. Additionally, the effectiveness of a genetic search when applied to improve the worst performing concept descriptions is also presented.