In this thesis we investigate the relationships between the symbolic and sub-symbolic methods used for controlling agents by artificial intelligence, focusing in particular on methods that learn. In light of the strengths and weaknesses of each approach, we propose a hybridisation of symbolic and subsymbolic methods to capitalise on the best features of each. We implement such a hybrid system, called Rachel which incorporates techniques from Teleo-Reactive Planning, Hierarchical Reinforcement Learning and Inductive Logic Programming. Rachel uses a novel representation of be-haviours, Reinforcement-Learnt Teleo-operators (RL-Tops), which defines the behaviour in terms of its desired consequences but leaves the implementation of the policy to be learnt by reinforcement learning. An RL-Top is an abstract, symbolic description of the purpose of a behaviour, and is used by Rachel both as a planning operator and as the definition of a reward function by which the behaviour can be learnt. Two new

Introduction Much machine learning research addresses inductive learning --- learning relationships from a set of examples (Michalski (1986) provides an excellent introduction). For instance, some programs have been used to learn medical diagnostic rules from a database of patients whose diagnoses are known. These programs examine a number of attributes (e.g. age, temperature, and pulse rate) for a set of examples whose classification (e.g. diagnosis) is known. This set of examples is termed a training set. Attributes tests are combined into logical rules which are used to predict the classification (e.g. if (age > 5) and (temperature > 100), then (preliminary-diagnosis = not-normal)). These rules are generally termed concepts. Reliability and induction The probability that a given concept will accurately classify a training set by chance alone, denoted here as P, is a fundamental cha

Abstract. In this paper, we propose a new attribute weight setting method for k-NN based classifiers using quadratic programming, which is particular suitable for binary classification problems. Our method formalises the attribute weight setting problem as a quadratic programming problem and exploits commercial software to calculate attribute weights. Experiments show that our method is quite practical for various problems and can achieve a competitive performance. Another merit of the method is that it can use small training sets. 1.

this paper. We maintain that any viable theory of human learning must be based on this definition, and we will see that many common learning methods satisfy it, though they are seldom presented in these terms. We can loosen our definition somewhat to allow storage of a few competing knowledge structures, or to allow a current structure with a number of possible successors, from which one is then selected. These variations still restrict memory to a manageable size. 2.2 Definitions of Order Effects

Abstract. The k-Nearest Neighbour (k-NN) method is a typical lazy learning paradigm for solving classification problems. Although this method was originally proposed as a non-parameterised method, attribute weight setting has been commonly adopted to deal with irrelevant attributes. In this paper, we propose a new attribute weight setting method for k-NN based classifiers using quadratic programming, which is particularly suitable for binary classification problems. Our method formalises the attribute weight setting problem as a quadratic programming problem and exploits commercial software to calculate attribute weights. To evaluate our method, we carried out a series of experiments on six established data sets. Experiments show that our method is quite practical for various problems and can achieve a stable increase in accuracy over the standard k-NN method as well as a competitive performance. Another merit of the method is that it can use small training sets.

j_ REPORT DOCUMENTATION PAGE OMBNo.07o4o188 Oijt31!C _e_3rt,r1 _ burden +c,r this, oile(tlOtl]f,nformatlOn,s estimated to average 1 hour per resooqse irlcIu_4_g rife t_me tor re_e'hmg irlstruGIOr;$, sear(rang e,rsDng dat _ sour¢_., _ather_r _:_r_d-na!_t_l_r_g the _a[a needed. 3nd Como/etlng 3nO reviewing lhe c3t{_,<_l©rl,)f rr f,_rma_l©n S_'nd:ommertt _ re_arctmg this burden estbma_e or _rly other asDe _ of l_s c,_lecliQt _ 3f mtormat_O r_. mc)udlrlg sugg_tl_n _, f_r r@d_ng thl' _ DurGen to,a/a_hl_gT,,Dr _ _4eaclGuarter _ Ser,_ces, Curectorate for mforma_lorl ODerat_on' _ and ReoK)r%, 12'15 Je_fE, r_on

this paper, we present a joint concept formation system, SGNN, that extends the previous work of concept formation [Fisher, 1987; McKusick and Langley, 1991]. SGNN is able to generate either disjoint concept trees or acyclic directed concept graphs, according to the characteristics inherited in domain data. Furthermore, with certain controls applied to the number of winners at each concept layer, SGNN can also be used to only construct disjoint concept trees regardless of the regularity inherited in data. It is demonstrated 15

We introduce coactive learning as a distributed learning approach to data mining in networked and distributed databases. The coactive learning algorithms act on independent data sets and cooperate by communicating training information, which is used to guide the algorithms' hypothesis construction. The exchanged training information is limited to examples and responses to examples. It is shown that coactive learning can offer a solution to learning on very large data sets by allowing multiple coacting algorithms to learn in parallel on subsets of the data, even if the subsets are distributed over a network. Coactive learning supports the construction of global concept descriptions even when the individual learning algorithms are provided with training sets having biased class distributions. Finally, the capabilities of coactive learning are demonstrated on artificial noisy domains, and on real world domain data with sparse class representation and unknown attribute valu...

The paper describes a hybrid inductive machine learning algorithm called CLIP4. The algorithm first partitions data into subsets using a tree structure and then generates production rules only from subsets stored at the leaf nodes. The unique feature of the algorithm is generation of rules that involve inequalities. The algorithm works with the data that have large number of examples and attributes, can cope with noisy data, and can use numerical, nominal, continuous, and missing-value attributes. The algorithm's flexibility and e#ciency are shown on several well-known benchmarking data sets, and the results are compared with other machine learning algorithms. The benchmarking results in each instance show the CLIP4's accuracy, CPU time, and rule complexity. CLIP4 has built-in features like tree pruning, methods for partitioning the data (for data with large number of examples and attributes, and for data containing noise), dataindependent mechanism for dealing with missing values, genetic operators to improve accuracy on small data, and the discretization schemes. CLIP4 generates model of data that consists of well-generalized rules, and ranks attributes and selectors that can be used for feature selection.

Most human concepts are flexible in the sense that they inherently lack: precise boundaries, and these boundaries are often contextdependent. This chapter describes 'a method for representing and inductively learning flexible concepts from examples. The basic idea is to represent such concepts using a two-tiered representation.. Such a representation consists of two structures ("tiers"): the Base Concept Representation (BCR), which captures explicitly the basic and context- independent concept properties, and Inferential Concept Interpretation (ICI), which :haracterizes allowable concept modifications and contextdependency. The proposed method has been implemented in the POSEIDON 3 system (also called AQ16), and tested on various practical problems, such as learning the concept of "Acceptable union contracts" and "Voting patterns of Republicans and Democrats in the U.S. Congress." In the experiments, the system generated concept descriptions that were both, more accurate and simpler than those produced by other methods tested, such as methods employing simple exemplar-based representations, decision tree learning, and some previous methods for rule learning.