In this paper we present 1BC, a first-order Bayesian Classifier. Our approach is to view individuals as structured terms, and to distinguish between structural predicates referring to subterms (e.g. atoms from molecules), and properties applying to one or several of these subterms (e.g. a bond between two atoms). We describe an individual in terms of elementary features consisting of zero or more structural predicates and one property; these features are considered conditionally independent following the usual naive Bayes assumption. 1BC has been implemented in the context of the first-order descriptive learner Tertius, and we describe several experiments demonstrating the viability of our approach.

. This paper deals with learning first-order logic rules from data lacking an explicit classification predicate. Consequently, the learned rules are not restricted to predicate definitions as in supervised inductive logic programming. First-order logic offers the ability to deal with structured, multi-relational knowledge. Possible applications include first-order knowledge discovery, induction of integrity constraints in databases, multiple predicate learning, and learning mixed theories of predicate definitions and integrity constraints. One of the contributions of our work is a heuristic measure of confirmation, trading off novelty and satisfaction of the rule. The approach has been implemented in the Tertius system. The system performs an optimal bestfirst search, finding the k most confirmed hypotheses, and includes a non-redundant refinement operator to avoid duplicates in the search. Tertius can be adapted to many different domains by tuning its parameters, and it can deal eithe...

In this paper we present 1BC and 1BC2, two systems that perform naive Bayesian classification of structured individuals. The approach of 1BC is to project the individuals along first-order features. These features are built from the individual using structural predicates referring to related objects (e.g. atoms within molecules), and properties applying to the individual or one or several of its related objects (e.g. a bond between two atoms). We describe an individual in terms of elementary features consisting of zero or more structural predicates and one property; these features are treated as conditionally independent in the spirit of the naive Bayes assumption. 1BC2 represents an alternative first-order upgrade to the naive Bayesian classifier by considering probability distributions over structured objects (e.g., a molecule as a set of atoms), and estimating those distributions from the probabilities of its elements (which are assumed to be independent). We present a unifying view on both systems in which 1BC works in language space, and 1BC2 works in individual space. We also present a new, efficient recursive algorithm improving upon the original propositionalisation approach of 1BC. Both systems have been implemented in the context of the first-order descriptive learner Tertius, and we investigate the differences between the two systems both in computational terms and on artificially generated data. Finally, we describe a range of experiments on ILP benchmark data sets demonstrating the viability of our approach.

Abstract not found

This paper proposes a unifying framework for inductive rule learning algorithms. We suggest that the problem of constructing an appropriate inductive hypothesis (set of rules) can be broken down in the following subtasks: rule construction, body construction, and feature construction. Each of these subtasks may have its own declarative bias, search strategies, and heuristics. In particular, we argue that feature construction is a crucial notion in explaining the relations between attribute-value rule learning and inductive logic programming (ILP). We demonstrate this by a general method for transforming ILP problems to attributevalue form, which overcomes some of the traditional limitations of propositionalisation approaches.

This paper details the implementation of a strongly-typed evolutionary programming system (STEPS) and its application to concept learning from highly-structured examples. STEPS evolves concept descriptions in the form of program trees. Predictive accuracy is used as the fitness function to be optimised through genetic operations. Empirical results with representative applications demonstrate promise. 1 Introduction The aim of concept learning is to induce a general description of a concept from a set of specific examples. The examples and the concept description are expressed in some representation language (e.g., attribute-value language, Horn clauses) and the learning task can be viewed as a search, through the space of all possible concept descriptions, for a description that both characterises the examples provided and generalises to new ones [9]. As concept learning problems of increasing complexity are being tackled, increasingly expressive representation languages are becoming...

Computational logic has already significantly influenced (symbolic) machine learning through the field of inductive logic programming (ILP) which is concerned with the induction of logic programs from examples and background knowledge. In ILP, the shift of attention from program synthesis to knowledge discovery resulted in advanced techniques that are practically applicable for discovering knowledge in relational databases. Machine learning, and ILP in particular, has the potential to influence computational logic by providing an application area full of industrially significant problems, thus providing a challenge for other techniques in computational logic. This paper gives a brief introduction to ILP, presents state-of-the-art ILP techniques for relational knowledge discovery as well as some research and organizational directions for further developments in this area. 1 Introduction Inductive logic programming (ILP) [35, 39, 29] is a research area that has its backgrounds in induct...

. In this paper we present 1BC, a first-order Bayesian Classifier. Our approach is to view individuals as structured objects, and to distinguish between structural predicates referring to parts of individuals (e.g. atoms within molecules), and properties applying to the individual or one or several of its parts (e.g. a bond between two atoms). We describe an individual in terms of elementary features consisting of zero or more structural predicates and one property; these features are considered conditionally independent following the usual naive Bayes assumption. 1BC has been implemented in the context of the first-order descriptive learner Tertius, and we describe several experiments demonstrating the viability of our approach. Keywords: inductive logic programming, naive Bayes, first-order logic 1. Motivation and scope In this paper we present 1BC, a first-order Bayesian Classifier. While the propositional Bayesian Classifier makes the naive Bayes assumption of statistical indepe...

This paper provides a foundation for inductive learning based on the use of higherorder logic for knowledge representation. In particular, the paper (i) provides a systematic individuals-as-terms approach to knowledge representation for inductive learning, and demonstrates the utility of types and higher-order constructs for this purpose; (ii) gives a systematic way of constructing predicates for use in induced definitions; (iii) widens the applicability of decision-tree algorithms beyond the usual attribute-value setting to the classification of individuals with complex structure; and (iv) shows how to induce definitions which are comprehensible and have predictive power. The paper contains ten illustrative applications involving a variety of types to which a decision-tree learning system is applied. The e#ectiveness of the approach is further demonstrated by applying the learning system to two larger benchmark applications. 1 Introduction Inductive learning focuses on tec...

Naive Bayesian classifiers have been very successful in attribute-value representations.