Abstract This dissertation investigates the field of inductive logic programming (ILP) and in so doing an ILP system, Lime, is designed and developed. Lime addresses the problem of noisy training examples; learning from only positive, only negative, or both positive and negative examples; efficiently biasing and searching the hypothesis space; and handling recursion efficiently and effectively. The Q-heuristic is introduced to address the problem of learning with both noisy training examples and fixed numbers of positive and negative training examples. This heuristics is based on Bayes rule. Both a justification of its derivation and a description of the context in which it is appropriately applied are given. Because of the general nature of this heuristic its application is not restricted to ILP. Instead of employing a greedy covering approach to constructing clauses, Lime employs the Qheuristic to evaluate entire logic programs as hypotheses. To tame the inevitable explosion in the search space, the notion of a simple clause is introduced. These sets of literals may be viewed as subparts of clauses that are effectively independent in terms of variables used. Instead of growing a clause one literal at a time, Lime efficiently combines simple clauses to construct a set of gainful candidate clauses. Subsets of these candidate clauses are evaluated using the Q-heuristic to find the final hypothesis. Details of the algorithms and data structures of Lime are discussed. Lime's handling of recursive logic programs is also described. Experimental results are provided to illustrate how Lime achieves its design goals of better noise handling, learning from a fixed set of examples (e.g., from only positive data), and of learning recursive logic programs. These results compare the performance of Lime with other leading ILP systems like Foil and Progol in a variety of domains. Empirical results with a boosted version of Lime are also reported.

. Integrity constraints are important logical tools for the general organization of knowledge. Integrity constraints (in short: ICs), which are commonly used in the field of deductive databases, specify general regularities like "a son is not older than his father." They facilitate the organization of knowledge in expert systems and can speed up the queryresponse time significantly. This paper presents an approach for inductively generating compact integrity constraints from knowledge bases, represented in first-order logic. To obtain the most powerful ICs, the huge space of potential ICs, which are principally consistent with a given knowledge base, is restricted by IC-schemes. IC-schemes specify ICs syntactically. The proposed method searches the resulting space of ICs efficiently by pruning away whole subspaces. The approach is also capable of detecting irregularities in "noisy" knowledge bases which might be inconsistent. Empirical results illustrate the appropriateness of this me...

ion of the Inference Structure If the user selects the menu item Generate Topology, the system does that, using the algorithms specified in the parameter pst generating algorithms. 9.2 Topology-influenced Learning Mobal's learning tool Rdt is a model-based algorithm. The is restricted by: ffl rule models, ffl the arity compatibility of the predicates and the predicate variables in the rule models, ffl the sort compatibilities, based on user entered sorts or argument sort build by Stt. There are two main reasons to restrict the hypothesis space by Pst. First, if the user enters a topology, he gives the system a task structure. Learning by induction always contains the danger of learning senseless rules, if there is a basis for these rules in the knowledge base. Thus it is wise to use the given task structure to prevent the generation of senseless hypotheses by allowing only those that are compatible with the task structure. Second, the restriction of the hypothesis space reduces th...

Inductive learning in relational domains has been shown to be intractable in general. Many approaches to this task have been suggested nevertheless; all in some way restrict the hypothesis space searched. They can be roughly divided into two groups: data-driven, where the restriction is encoded into the algorithm, and model-based, where the restrictions are made more or less explicit with some form of declarative bias. This paper describes Incy, an inductive learner that seeks to combine aspects of both approaches. Incy is initially data-driven, using examples and background knowledge to put forth and specialize hypotheses based on the "connectivity" of the data at hand. It is model-driven in that hypotheses are abstracted into rule models, which are used both for control decisions in the data-driven phase and for model-guided induction. Key Words: Inductive learning in relational domains, cooperation of data-driven and model-guided methods, implicit and declarative bias. 1 Introduc...

. We present a general approach to combine methods of interactive knowledge acquisition with methods for machine learning. The approach has been developed in order to deliver knowledge required by support-systems for design-tasks. Learning rests upon a knowledge representation scheme for cases that distinguishes between knowledge needed for subgoaling and knowledge needed for design. We employ traces, i.e., protocols of the user's actions when tackling design-tasks as the initial input for incremental knowledge acquisition. This allows to learn task structures to be used for subgoaling and case-bases plus similarity relations applicable to particular case-bases. 1 INTRODUCTION Integrating incremental learning into a knowledge-based systems seems to be a promising way to lessen the burden of knowledge elicitation to system development [9]. The goal of this paper is to point out how learning can be used in an interactive design-support system that uses Cbr [8] as the main problem solvin...

ion of the Inference Structure If the user selects the menu item Generate Topology, the system does that, using the algorithms specified in the parameter pst generating algorithms. 10.2 Topology-influenced Learning Mobal's learning tool Rdt is a model-based algorithm. The is restricted by: ffl rule models, ffl the arity compatibility of the predicates and the predicate variables in the rule models, 10 PST: THE PREDICATE STRUCTURING TOOL 65 ffl the sort compatibilities, based on user entered sorts or argument sort build by Stt. There are two main reasons to restrict the hypothesis space by Pst. First, if the user enters a topology, he gives the system a task structure. Learning by induction always contains the danger of learning senseless rules, if there is a basis for these rules in the knowledge base. Thus it is wise to use the given task structure to prevent the generation of senseless hypotheses by allowing only those that are compatible with the task structure. Second, the re...

The automatic construction of classifiers (programs able to correctly classify data collected from the real world) is one of the major problems in pattern recognition and in a wide area related to artificial intelligence, including data mining. In this paper, we present G-Net, a distributed evolutionary algorithm able to infer classifiers from precollected data. The main features of the system include robustness with respect to parameter settings, use of the minimum description length (MDL) criterion coupled with a stochastic search bias, coevolution as high-level control strategy, ability to face problems requiring structured representation languages, and suitability to parallel implementation on a network of workstations (NOW). Its parallel version, NOW G-Net, also described in this paper, is able to profitably exploit the computing power delivered by these platforms by incorporating a set of dynamic load distribution techniques that allow it to adapt to the variations of computing power arising typically in these systems. A proof-of-concept implementation based on PVM is used in the paper to demonstrate the effectiveness of NOW G-Net on a variety of datasets.

This report demonstrates the eectiveness of logic minimization in learning from examples. Initially the paper reviews logic minimization and relates it with learning. To support logic minimization we present a system (called LML), the core of which derives from the implementation of the Espresso-II algorithm (Brayton et al., 1984). Espresso-II is popular in VLSI synthesis and design. We show that logic minimization extends the general logic diagram approach as used to support conceptual clustering (Michalski & Stepp, 1983) and diagrammatic visualization of concepts (Wnek & Michalski, 1994) in learning from examples. We test our approach using two toy domains and ten real world domains. We discuss search space taken into account by logic minimization. Furthermore, we compare performance of LML with C4.5, AQ15, NewId and CN2 using classication accuracy, rule quality, and draw curves with respect to the number of training examples required for learning. We conclude our work by linking L

The Semantic Web relies heavily on the formal ontologies that structure underlying data for the purpose of comprehensive and transportable machine understanding. Therefore, the success of the Semantic Web depends strongly on the proliferation of ontologies, which requires fast and easy engineering of ontologies and avoidance of a knowledge acquisition bottleneck. Ontology Learning greatly facilitates the construction of ontologies by the ontology engineer. The vision of ontology learning that we propose here includes a number of complementary disciplines that feed on different types of unstructured, semi-structured and fully structured data in order to support a semi-automatic, cooperative ontology engineering process. Our ontology learning framework proceeds through ontology import, extraction, pruning, refinement, and evaluation giving the ontology engineer a wealth of coordinated tools for ontology modeling. Besides of the general framework and architecture, we show in thi...

Contents Acknowledgments iii 1 Introduction 1 1.1 Limits of ILP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Proposed Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Structure of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 An Overview of Machine Learning 7 2.1 Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1.1 Learning Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.1.2 Research Paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 Inductive Concept Learning from Examples . . . . . . . . . . . . . . . . . . . 10 2.3 Representation Languages in Inductive Reasoning . . . . . . . . . . . . . . . . 12 3 Inductive Logic Programming 17 3.1 Logic Programming Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.2 Learning from Entailment . . . . . . . . . . . . . . . . . . . . . . . . . . . . .