Abstract. This paper presents a Bayesian method for constructing probabilistic networks from databases. In particular, we focus on constructing Bayesian belief networks. Potential applications include computer-assisted hypothesis testing, automated scientific discovery, and automated construction of probabilistic expert systems. We extend the basic method to handle missing data and hidden (latent) variables. We show how to perform probabilistic inference by averaging over the inferences of multiple belief networks. Results are presented of a preliminary evaluation of an algorithm for constructing a belief network from a database of cases. Finally, we relate the methods in this paper to previous work, and we discuss open problems.

The main contributions of this thesis are a Bayesian theory of learning classification rules, the unification and comparison of this theory with some previous theories of learning, and two extensive applications of the theory to the problems of learning class probability trees and bounding error when learning logical rules. The thesis is motivated by considering some current research issues in machine learning such as bias, overfitting and search, and considering the requirements placed on a learning system when it is used for knowledge acquisition. Basic Bayesian decision theory relevant to the problem of learning classification rules is reviewed, then a Bayesian framework for such learning is presented. The framework has three components: the hypothesis space, the learning protocol, and criteria for successful learning. Several learning protocols are analysed in detail: queries, logical, noisy, uncertain and positive-only examples. The analysis is done by interpreting a protocol as a...

Recently, there has been an increased interest in "lifelong " machine learning methods, that transfer knowledge across multiple learning tasks. Such methods have repeatedly been found to outperform conventional, single-task learning algorithms when the learning tasks are appropriately related. To increase robustness of such approaches, methods are desirable that can reason about the relatedness of individuallearning tasks, in order to avoid the danger arising from tasks that are unrelated and thus potentially misleading. This paper describes the task-clustering (TC) algorithm. TC clusters learning tasks into classes of mutually related tasks. When facing a new learning task, TC first determines the most related task cluster, then exploits information selectively from this task cluster only. An empirical study carried out in a mobile robot domain shows that TC outperforms its non-selective counterpart in situations where only a small number of tasks is relevant. 1 INTRODUCTION One of t...

Digital libraries of images and video are rapidly growing in size and availability. To avoid the expense and limitations of text, there is considerable interest in navigation by perceptual and other automatically extractable attributes. Unfortunately, the relevance of an attribute for a query is not always obvious. Queries which go beyond explicit color, shape, and positional cues must incorporate multiple features in complex ways. This dissertation uses machine learning to automatically select and combine features to satisfy a query, based on positive and negative examples from the user. The learning algorithm does not just learn during the course of one session: it learns continuously, across sessions. The learner improves its learning ability by dynamically modifying its inductive bias, based on experience over multiple sessions. Experiments demonstrate the ability to assist image classification, segmentation, and annotation (labeling of image regions). The common theme of this work...

It is widely appreciated (e.g. Marr, 1982) that the difficulty of a particular computation varies according to how the input data are presented. What is less well understood is the effect of this computation/representation trade-off within familiar learning paradigms. We argue that existing learning algorithms are often poorly equipped to solve problems involving a certain type of important and widespread statistical regularity, which we call `type-2 regularity'. The solution in these cases is to trade achieved representation against computational search. We investigate several ways in which such a trade-off may be pursued including simple incremental learning, modular connectionism, and the developmental hypothesis of `representational redescription'. In addition, the most distinctive features of human cognition --- language and culture --- may themselves be viewed as adaptations enabling this representation/computation trade-off to be pursued on an even grander scale.

The use of bias with automated learning systems has become an important area of research. The use of bias with evolutionary techniques of learning has been shown to have advantages when complex structures are evolved. This is especially true when the semantics of the evolving population of structures is not explicitly represented or analysed during the evolution. This paper describes a framework which brings together two types of bias, namely search bias (the way new structures are created) and language bias (the form of possible structures that may be created). The resulting system extends genetic programming (GP) by allowing declarative bias with both the form of possible solutions that are created and the method by which they are transformed. 1 Introduction "All our experiences in AI research have led us to believe that for automatic programming, the answer lies in knowledge, in adding a collection of expert rules which will guide code synthesis and transforma...

Recently, there has been an increased interest in machine learning methods that learn from more than one learning task. Such methods have repeatedly found to outperform conventional, single-task learning algorithms when learning tasks are appropriately related. To increase robustness of these approaches, methods are desirable that can reason about the relatedness of individual learning tasks, in order to avoid the danger arising from tasks that are unrelated and thus potentially misleading. This paper describes the task-clustering (TC) algorithm. TC clusters learning tasks into classes of mutually related tasks. When facing a new thing to learn, TC first determines the most related task cluster, then exploits information selectively from this task cluster only. An empirical study carried out in a mobile robot domain shows that TC outperforms its unselective counterpart in situations where only a small number of tasks is relevant. 1 Introduction One of the exciting new developments i...

views and conclusionscontained in this documentare those of the author and should not be interpreted as necessarily representing official policies or endorsements, either expressed or implied, of NSF, Wright Laboratory or the United States Government. Keywords: Artificial neural networks, bias, concept learning, knowledge transfer, lifelong learning, machine learning, object recognition, relevance, supervised Machine learning has not yet succeeded in the design of robust learning algorithms that generalize well from very small datasets. In contrast, humans often generalize correctly from only a single training example, even if the number of potentially relevant features is large. To do so, they successfully exploit knowledge acquired in previous learning tasks, to bias subsequent learning. This paper investigates learning in a lifelong context. Lifelong learning addresses situations where a learner faces a stream of learning tasks. Such scenarios provide the opportunity for synergetic effects that arise if knowledge is transferred across multiple learning tasks. To study the utility of transfer, several approaches to lifelong learning are proposed and evaluated in an object recognition domain. It

Each concept description language and search strategy has an inherent inductive bias, a preference for some hypotheses over others. No single inductive bias performs optimally on all problems. This paper describes a system that couples induction with optimization to carry out an efficient search of large regions of inductive bias space. Experimental results are reported demonstrating the system's capacity to choose optimal biases even for complex and noisy problems. 1

Many engineering problems may be described as a search for one near optimal description amongst many possibilities, given certain constraints. Search techniques, such as genetic programming, seem appropriate to represent many problems. This paper describes a grammatically based learning technique, based upon the genetic programming paradigm, that allows declarative biasing and modifies the bias as the evolution proceeds. The use of bias allows complex problems to be represented and searched efficiently. 1 Introduction The Genetic Programming paradigm (GP) is a form of adaptive learning [4]. The technique is based upon the genetic algorithm (GA), [2], which exploits the process of natural selection based on a fitness measure to breed a population that improves over time. The ability of GA's to efficiently search large conceptual spaces makes them suitable for the discovery and induction of generalisations from a data set. A summary of the genetic programming paradigm may be found in [6...