We present MLC++, a library of C++ classes and tools for supervised Machine Learning. While MLC++ provides general learning algorithms that can be used by end users, the main objective is to provide researchers and experts with a wide variety of tools that can accelerate algorithm development, increase software reliability, provide comparison tools, and display information visually. More than just a collection of existing algorithms, MLC++ is an attempt to extract commonalities of algorithms and decompose them for a unified view that is simple, coherent, and extensible. In this paper we discuss the problems MLC++ aims to solve, the design of MLC++, and the current functionality.

Abstract. Supervised learning in attribute-based spaces is one of the most popular machine learning problems studied and, consequently, has attracted considerable attention of the genetic algorithm community. The fullmemory approach developed here uses the same nigh-level descriptive language that is used in rule-based systems. This allows for an easy utilization of inference rules of the well-known inductive learning methodology, which replace the traditional domain-independent operators and make the search task-specific. Moreover, a closer relationship between the underlying task and the processing mechanisms provides a setting for an application of more powerful task-specific heuristics. Initial results obtained with a prototype implementation for the simplest case of single concepts indicate that genetic algorithms can be effectively used to process nigh-level concepts and incorporate task-specific knowledge. The method of abstracting the genetic algorithm to the problem level, described here for the supervised inductive learning, can be also extended to other domains and tasks, since it provides a framework for combining recently popular genetic algorithm methods with traditional problem-solving methodologies. Moreover, in this particular case, it provides a very powerful tool enabling study of the widely accepted but not so well understood inductive learning methodology.

Thedevelopmentofmultistrategylearningsystemsrequiresaclearunderstandingoftherolesandthe applicabilityconditionsofdifferentlearningstrategies.Tothisend,thischapterintroducesthe InferentialTheoryofLearning thatprovidesaconceptualframeworkforexplaininglogicalcapabilities oflearningstrategies,i.e.,their competence.Viewinglearningasaprocessofmodifyingthelearner's knowledgebyexploringthelearner'sexperience,thetheorypostulatesthatanysuchprocesscanbe describedasasearchina knowledgespace, which involvesthelearner'sexperience,piorknowledgeand the learninggoal .Thesearchoperatorsareinstantiationsof knowledgetransmutations, whichare genericpatternsofknowledgechange.Transmutationsmayemployanybasictypeofinference --- deduction,inductionoranalogy.Severalfundamentalknowledg etransmutationsaredescribedinanovel andgeneralway,suchasgeneralization,abstraction,explanationandsimilization,andtheircounterparts, specialization,concretion,predictionanddissimilization,respectively.Generalizationenlargesthe referenceset ofadescription(thesetofentitiesthatarebeingdescribed).Abstractionreducesthe amountofthedetailaboutthereferenceset.Explanationgeneratespremisesthatexplain(orimply)the givenpropertiesofthereferenceset.Similization transfersknowledgefromonereferencesettoasimilar referenceset.Usingconceptsofthetheory,a multistrategytask -adaptivelearning(MTL)methodology isoutlined,andillustratedbyanexample.MTLdynamicallyadaptsstrategiestothe learningtask , definedbytheinputinformation,learner'sbackgroundknowledge,andthelearninggoal. Thegoalof MTLresearchisto synergisticallyintegrateawiderangeofinferentiallearningstrategies,suchas empiricalgeneralization,constructiveinduction, deductivegeneralization,explanation,prediction, abstraction,andsimilization. Keywords: learningtheory,inferencetheory,multi...

This paper describes a method for learning flexible concepts. by which are meant concepts that lack precise definition and are contextqlependent. To describe such concepts, the method employs a two-tiered represen- tation. in which the first tier captures explicitly basic concept properties, and the second tier characterizes allowable concept's modifications and context dependency. In e proposed method. the first tier, called Base Concept Representation (BCR), is created in two phases. In phase 1, the AQ-15 rule learning program is applied to induce a complete and consistent concept description from supplied examples. In phase 2, this description is optimized according to a domain-dependent quality criterion. The second tier, called the inferential concept interpretation dCI). consists of a procedure for flexible matching, and a set of inference rules. The proposed method has been implemented in the POSEIDON system. and experimentally tested on two real-world problems: [earning the concept of an acceptable umon contract, and learning voting patterns of Republicans and Democrats in the U.S. Congress. For comparison, a few other learning methods were also applied to the same problems. These methods included simple variants of exemplar-based learning, and an ID-3-tyl: decision tree learning, implemented m the ASSISTANT program. In the exl:riments, POSEIDON generated concept descriptions that were both, more accurate and also substantially simpler than those produced by the other methods.

This paper discusses the unsupervised learning problem. An important part of the unsupervised learning problem is determining the number of constituent groups (components or classes) which best describes some data. We apply the Minimum Message Length (MML) criterion to the unsupervised learning problem, modifying an earlier such MML application. We give an empirical comparison of criteria prominent in the literature for estimating the number of components in a data set. We conclude that the Minimum Message Length criterion performs better than the alternatives on the data considered here for unsupervised learning tasks.

In this introduction, we define the term bias as it is used in machine learning systems. We motivate the importance of automated methods for evaluating and selecting biases using a framework of bias selection as sem'ch in bias and meta-bias spaces. Recent research in the field of mac}fine learning bias is stmmarized.

This paper presents a method for constructive induction, in which new attributes are constructed as various functions of original attributes. Such a method is called data-driven constructive induction, because new attributes are derived from an analysis of the data (examples) rather than the generated rules. Attribute construction and rule generation is repeated until a termination condition, such as the satisfaction of a rule quality measure, is met. The first step of this method, the generation of new attributes has been implemented in AQ17-PRE. Initial experiments with AQ17-PRE have shown that it leads to an improvement of the learned rules both in terms of their

An enormous proliferation of databases in almost every area of human endeavor has created a great demand for new, powerful tools for turning data into useful, task-oriented knowledge. In efforts to satisfy this need, researchers have been exploring ideas and methods developed in machine learning, pattern recognition, statistical data analysis, data visualization, neural nets, etc. These efforts have led to the emergence of a new research area, frequently called data mining and knowledge discovery. The first part of this chapter is a compendium of ideas on the applicability of symbolic machine learning methods to this area. The second part describes a multistrategy methodology for conceptual data exploration, by which we mean the derivation of high-level concepts and descriptions from data through symbolic reasoning involving both data and background knowledge. The methodology, which has been implemented in the INLEN system, combines machine learning, database and knowledge-based techn...

This paper presents a method for multistrategy constructive induction that integrates two inferential learning strategies—empirical induction and deduction, and two computational methods—data-driven and hypothesis-driven. The method generates inductive hypotheses in an iteratively modified representation space. The operators modifying the representation space are classified into "constructors, " which expand the space (by generating additional attributes) and "destructors " which contract the space (by removing low relevance attributes or abstracting attribute values). Constructors generate new dimensions (attributes) by analyzing original or transformed examples (data-driven) and by analyzing the rules obtained in the previous iteration (hypothesisdriven). Destructors detect the irrelevant components of the representation space by rulebased inference or statistical analysis. The method has been implemented in the AQ17-MCI program. The preliminary results from applying it to a problem with noisy training data and large number of irrelevant attributes demonstrated a superiority of the method over other constructive induction methods both in terms of the predictive accuracy, as well as the overall simplicity of the generated descriptions. Key words: multistrategy learning, inductive inference, constructive induction, representation space, concept learning. 1.

Feature construction through aggregation plays an essential role in modeling relational domains with one-to-many relationships between tables. One-to-many relationships lead to bags (multisets) of related entities, from which predictive information must be captured. This paper focuses on aggregation from categorical attributes that can take many values (e.g., object identifiers). We present a novel aggregation method as part of a relational learning system ACORA, that combines the use of vector distance and meta-data about the class-conditional distributions of attribute values. We provide a theoretical foundation for this approach deriving a “relational fixed-effect ” model within a Bayesian framework, and discuss the implications of identifier aggregation on the expressive power of the induced model. One advantage of using identifier attributes is the circumvention of limitations caused either by missing/unobserved object properties or by independence assumptions. Finally, we show empirically that the novel aggregators can generalize in the presence of identifier (and other high-dimensional) attributes, and also explore the limitations of the applicability of the methods. 1