This paper contains a systematic study of the foundations of knowledge representation, computation, and learning in higher-order logic. First, a polymorphically-typed higher-order logic, whose origins can be traced back to Church's simple theory of types, is presented. A model theory and proof theory for this logic are developed and basic theorems relating these two are given. A metric space of certain closed terms, which provides a rich language for representing individuals, is then studied. Also a method of systematically constructing predicates on such individuals is given. The technique of programming with abstractions is illustrated. Major applications of the logic to declarative programming languages and machine learning are indicated. 1

Abstract. This paper studies the PAC and agnostic PAC learnability of some standard function classes in the learning in higher-order logic setting introduced by Lloyd et al. In particular, it is shown that the similarity between learning in higher-order logic and traditional attributevalue learning allows many results from computational learning theory to be ‘ported ’ to the logical setting with ease. As a direct consequence, a number of non-trivial results in the higher-order setting can be established with straightforward proofs. Our satisfyingly simple analysis provides another case for a more in-depth study and wider uptake of the proposed higher-order logic approach to symbolic machine learning. 1

Abstract. We study predicate selection functions (also known as splitting rules) for structural decision trees and propose two improvements to existing schemes. The first is in classification learning, where we reconsider the use of accuracy as a predicate selection function and show that, on practical grounds, it is a better alternative to other commonly used functions. The second is in regression learning, where we consider the standard mean squared error measure and give a predicate pruning result for it. 1

Introduction This extended abstract outlines a submission to The Predictive Toxicology Challenge for 20002001 [PTC]. The Challenge is to obtain models that predict the outcome of biological tests for the toxicity of chemicals using information related to chemical structure only. The models reported here are based on the approach outlined in [BGCL00]. (Much more detail is given in [BGCL01].) In essence, the learning system is a decision-tree system. However, it is rather more general than conventional decision-tree learners in that it allows the individuals that are to be classified to be represented as certain terms in a higher-order logic rather than as simple feature vectors. The logic allows the use of sets, multisets, lists, graphs and so on, to represent individuals, thereby capturing complex structural information about the individuals [Llo01]. This information is highly relevant to the main aim of the Challenge, which is to predict toxicity from the chemical structure

Abstract not found

This paper describes the design and analysis of a system developed to learn comprehensible theories from structured data. The underlying knowledge representation formalism is a polymorphically-typed higher-order logic. To model structured data, a class of terms suitable for representing a wide range of data is identified. To encode structural boolean features, a class of predicates that can be built up by composition of simple functions is introduced. For any particular application, we give a mechanism to define and enumerate a set of relevant predicates. To construct comprehensible theories, we adopt the family of decision-tree algorithms, which include the standard top-down induction algorithm for learning decision trees, and the covering algorithm for learning decision lists. We show how these algorithms can be extended to work with structured data and complex predicates in both the batch and on-line settings. Using classical learning-theoretic results, we analyse the generalisation performance of our algorithms. The utility of the system is demonstrated with applications in the areas of bioinformatics and intelligent agents.