This paper is a survey of inductive rule learning algorithms that use a separate-and-conquer strategy. This strategy can be traced back to the AQ learning system and still enjoys popularity as can be seen from its frequent use in inductive logic programming systems. We will put this wide variety of algorithms into a single framework and analyze them along three different dimensions, namely their search, language and overfitting avoidance biases.

Declarative bias plays an important role when learning in potentially huge hypothesis spaces. While scientific discovery systems, which perform equation discovery as a subtask, consider such potentially huge hypothesis spaces, few (if any) employ declarative (as opposed to hard-coded) bias to define and restrict their hypothesis space. We present an equation discovery system Lagramge that uses grammars to define and restrict its hypothesis space. These grammars can make use of functions defined as domain specific knowledge, in addition to common mathematical operators. Lagramge was successfully applied to three artificial domains, rediscovering the correct equations. It was also applied to a real-world problem, discovering equations that make sense in terms of domain knowledge and produce accurate predictions. 1 INTRODUCTION The term bias refers to any kind of basis for choosing one generalization over another, other than strict consistency with the observed training examples [Mitchel...

. The state-of-the-art in inductive logic programming is surveyed by analyzing the approach taken by this field over the past 8 years. The analysis investigates the roles of 1) logic programming and machine learning, of 2) theory, techniques and applications, of 3) various technical problems addressed within inductive logic programming. 1 Introduction The term inductive logic programming was first coined by Stephen Muggleton in 1990 [1]. Inductive logic programming is concerned with the study of inductive machine learning within the representations offered by computational logic. Since 1991, annual international workshops have been organized [2-8]. This paper is an attempt to analyze the developments within this field. Particular attention is devoted to the relation between inductive logic programming and its neighboring fields such as machine learning, computational logic and data mining, and to the role that theory, techniques and implementations, and applications play. The analysis...

The past few years have witnessed an significant interest in probabilistic logic learning, i.e. in research lying at the intersection of probabilistic reasoning, logical representations, and machine learning. A rich variety of di#erent formalisms and learning techniques have been developed. This paper provides an introductory survey and overview of the stateof -the-art in probabilistic logic learning through the identification of a number of important probabilistic, logical and learning concepts.

The inductive synthesis of recursive logic programs from incomplete information, such as input/output examples, is a challenging subfield both of ILP (Inductive Logic Programming) and of the synthesis (in general) of logic programs from formal specifications. We first overview past and present achievements, focusing on the techniques that were designed specifically for the inductive synthesis of recursive logic programs, but also discussing a few general ILP techniques that can also induce non-recursive hypotheses. Then we analyse the prospects of these techniques in this task, investigating their applicability to software engineering as well as to knowledge acquisition and discovery.

. Interest in Declarative bias in Machine Learning is growing with the expressivity of the concept description language of ML systems. Inductive Logic Programming more than any other ML field is thus concerned with explicitely biasing learning. The main issues already identified in declarative bias [RG90] have been studied within the ILP project, i.e. the restriction of the size of the search space for the target concept and representation of the bias. As a first step, an extensive study of existing ILP systems and the elicitation of the role of hidden bias has led to define typologies of bias in relation with their effects on the learning process as well as alternative representation for bias. Declarative representations of bias have been defined through different types of languages so that bias can be easily set and shifted. In parallel with the definition, the representation and the experimentation of various biases, the interactions between different types of bias have been analyze...

This paper describes an active learning approach to the problem of grammatical inference, specifically the inference of deterministic finite automata (DFAs). We refer to the algorithm as the estimation-exploration algorithm (EEA). This approach differs from previous passive and active learning approaches to grammatical inference in that training data is actively proposed by the algorithm, rather than passively receiving training data from some external teacher. Here we show that this algorithm outperforms one version of the most powerful set of algorithms for grammatical inference, evidence driven state merging (EDSM), on randomly-generated DFAs. The performance increase is due to the fact that the EDSM algorithm only works well for DFAs with specific balances (percentage of positive labelings), while the EEA is more consistent over a wider range of balances. Based on this finding we propose a more general method for generating DFAs to be used in the development of future grammatical inference algorithms.

Induction of recursive theories in the normal ILP setting is a difficult learning task whose complexity is equivalent to multiple predicate learning. In this paper we propose computational solutions to some relevant issues raised by the multiple predicate learning problem. A separate-andparallel -conquer search strategy is adopted to interleave the learning of clauses supplying predicates with mutually recursive definitions. A novel generality order to be imposed on the search space of clauses is investigated, in order to cope with recursion in a more suitable way. The consistency recovery is performed by reformulating the current theory and by applying a layering technique, based on the collapsed dependency graph. The proposed approach has been implemented in the ILP system ATRE and tested on some laboratory-sized and real-world data sets. Experimental results demonstrate that ATRE is able to learn correct theories autonomously and to discover concept dependencies. Finally, related works and their main differences with our approach are discussed.

Discovery of frequent patterns has been studied in a variety of data mining (DM) settings. In its simplest form, known from association rule mining, the task is to find all frequent itemsets, i.e., to list all combinations of items that are found in a sufficient number of examples. A similar task in spirit, but at the opposite end of the complexity scale, is the Inductive Logic Programming (ILP) approach where the goal is to discover queries in first order logic that succeed with respect to a sufficient number of examples. We discuss the relationship of ILP to frequent pattern discovery. On one hand, our goal is to relate data mining problems to ILP. On another hand, we want to demonstrate how ILP can be used to solve both existing and new data mining problems. The fundamental task of association rule and frequent set discovery has been extended in various directions, allowing more useful patterns to be discovered. From an ILP viewpoint, however, it can be argued that these settings ar...

Stochastic logic programs combine ideas from probabilistic grammars with the expressive power of definite clause logic; as such they can be considered as an extension of probabilistic context-free grammars. Motivated by an analogy with learning tree-bank grammars, we study how to learn stochastic logic programs from proof-trees. Using proof-trees as examples imposes strong logical constraints on the structure of the target stochastic logic program. These constraints can be integrated in the least general generalization (lgg) operator, which is employed to traverse the search space. Our implementation employs a greedy search guided by the maximum likelihood principle and failure-adjusted maximization. We also report on a number of simple experiments that show the promise of the approach.