This article describes a comprehensive approach to automatic theory revision. Given an imperfect theory, the approach combines explanation attempts for incorrectly classified examples in order to identify the failing portions of the theory. For each theory fault, correlated subsets of the examples are used to inductively generate a correction. Because the corrections are focused, they tend to preserve the structure of the original theory. Because the system starts with an approximate domain theory, in general fewer training examples are required to attain a given level of performance (classification accuracy) compared to a purely empirical system. The approach applies to classification systems employing a propositional Horn-clause theory. The system has been tested in a variety of application domains, and results are presented for problems in the domains of molecular biology and plant disease diagnosis. 1 INTRODUCTION 2 1 Introduction One of the most difficult problems in the develo...

There are many learning problems for which the examples given by the teacher are ambiguously labeled. In this thesis, we will examine one framework of learning from ambiguous examples known as Multiple-Instance learning. Each example is a bag, consisting of any number of instances. A bag is labeled negative if all instances in it are negative. A bag is labeled positive if at least one instance in it is positive. Because the instances themselves are not labeled, each positive bag is an ambiguous example. We would like to learn a concept which will correctly classify unseen bags. We have developed a measure called Diverse Density and algorithms for learning from multiple-instance examples. We have applied these techniques to problems in drug design, stock prediction, and image database retrieval. These serve as examples of how to translate the ambiguity in the application domain into bags, as well as successful...

This chapter describes a multistrategy system that employs independent modules for deductive, abductive, and inductive reasoning to revise an arbitrarily incorrect propositional Horn-clause domain theory to t a set of preclassi ed training instances. By combining such diverse methods, Either is able to handle a wider range of imperfect theories than other theory revision systems while guaranteeing that the revised theory will be consistent with the training data. Either has successfully revised two actual expert theories, one in molecular biology and one in plant pathology. The results con rm the hypothesis that using a multistrategy system to learn from both theory and data gives better results than using either theory or data alone. 1

The paper presents an approach to the automation of knowledge acquisition for expert systems. The approach is based on several general principles emerging from the field of machine learning: expert system building as a three phase process, understanding-based knowledge extension, knowledge acquisition through multistrategy learning, consistency-driven concept formation and refinement, closed-loop learning. and synergistic cooperation between the human expert and the learning system. In this approach, an expert system is built by a human expert and a learning system. The human expert defmes the framework for the expert system and provides an incomplete and partially incorrect knowledge base. The learning system incrementally extends, updates, and improves the knowledge base through learning from the human expert. This approach is illustrated by the learning system shell NeoDISCIPLE.

. We present provably correct interactive algorithms for learning regular grammars from positive examples and membership queries. A structurally complete set of strings from a language L(G) corresponding to a target regular grammar G implicitly specifies a lattice of finite state automata (FSA) which contains a FSA MG corresponding to G. The lattice is compactly represented as a version-space and MG is identified by searching the version-space using membership queries. We explore the problem of regular grammar inference in a setting where positive examples are provided intermittently. We provide an incremental version of the algorithm along with a set of sufficient conditions for its convergence. 1 Introduction Regular Grammar Inference [3, 5, 9, 12] is an important machine learning problem with applications in pattern recognition and language acquisition. It is defined as the process of learning an unknown regular grammar (G) given a finite set of positive examples S + , possibly...

This paper presents an approach to learning from noisy data that views the problem as one of reasoning under uncertainty, where prior knowledge of the noise process is applied to compute a posteriori probabilities over the hypothesis space. In preliminary experiments this maximum a posteriori (MAP) approach exhibits a learning rate advantage over the C4.5 algorithm that is statistically significant. Introduction The classifier learning problem is to use a set of labeled training data to induce a classifier that will accurately classify as yet unseen, unclassified testing data. Some approaches assume that the training data is correct [ Mitchell, 1982 ] . Some assume that noise is present and simply tolerate it [ Breiman et al., 1984; Quinlan, 1987 ] . Another approach is to exploit knowledge of the presence and nature of noise [ Hirsh, 1990b ] . This paper takes the third approach, and views classifier learning from noisy data as a problem of reasoning under uncertainty, where knowle...

Because the knowledge discovery process is ill-defined, iterative, and requires intense interaction, algorithm flexibility is crucial. In this paper, we present a straighforward, heuristic generate-and-test search algorithm for knowledge discovery. An analysis of the literature shows that this basic algorithm underlies many of the systems that have had practical success in data mining and knowledge discovery over the past twenty years. We argue that this search algorithm has persevered because it is flexible and well behaved as background knowledge is introduced in various forms - exactly what is needed to support the ill-defined knowledge discovery process.

This paper presents an approach to concept learning from inconsistent data that foregoes a solution to the full-blown problem and instead considers a subcase, called bounded inconsistency. Data are said to have bounded inconsistency when some small perturbation to the description of any bad instance will result in a good instance.

Over the years we have developed the Disciple theory, methodology, and family of tools for building knowledge-based agents. This approach consists in developing an agent shell that can be taught directly by a subject matter expert, in a way that resembles how the expert would teach a human apprentice when solving problems in cooperation. This paper presents the most recent version of the Disciple approach and its implementation in the Disciple-RKF system. Disciple-RKF is based on methods for mixed-initiative problem solving, where the expert solves the more creative problems and the agent solves the more routine ones, integrated teaching and learning, where the agent helps the expert to teach it, by asking relevant questions, and the expert helps the agent to learn, by providing examples, hints and explanations, and multistrategy learning, where the agent integrates multiple learning strategies, such as learning from examples, learning from explanations, and learning by analogy, to learn from the expert how to solves problems. Disciple-RKF has been successfully applied to build learning and reasoning agents for military center of gravity analysis, which are used in several courses at the US Army War College. Key Words: multistrategy apprenticeship learning, problem solving through task reduction, mixed-initiative reasoning, plausible version spaces, rule learning, ontology, agent development, military center of gravity analysis 2 1.

Abstract. Machine learning techniques such as tree induction have become accepted tools for developing generalisations of large data sets, typically for use with production rule systems in prediction and classification. The advent of computer based cartography and the field of geographic information systems (GIS) has seen a wealth of spatial data generated and used for decision making and modelling. We examine the implications of inductive techniques applied to geospatial data in a logical framework. It is argued that spatial induction systems will benefit from the ability to extend their initial representation language, through feature and relation construction. The enormous search spaces involved imply a need for strong biasing techniques to control the generation of possible representations of the data for all but the most trivial of cases. A heavily constrained geospatial domain, topographic representation, is described as one simplified example of induction across a vector description of space. 1