This article describes a comprehensive approach to automatic theory revision. Given an imperfect theory, the approach combines explanation attempts for incorrectly classified examples in order to identify the failing portions of the theory. For each theory fault, correlated subsets of the examples are used to inductively generate a correction. Because the corrections are focused, they tend to preserve the structure of the original theory. Because the system starts with an approximate domain theory, in general fewer training examples are required to attain a given level of performance (classification accuracy) compared to a purely empirical system. The approach applies to classification systems employing a propositional Horn-clause theory. The system has been tested in a variety of application domains, and results are presented for problems in the domains of molecular biology and plant disease diagnosis. 1 INTRODUCTION 2 1 Introduction One of the most difficult problems in the develo...

This paper describes a successful approach to concept learning for heuristic classification. Almost all current programs for this task create or use explicit, abstract generalizations. These programs are largely ineffective for domains with weak or intractable theories. An exemplar-based approach is suitable for domains with inadequate theories but raises two additional problems: determining similarity and indexing exemplars. Our approach extends the exemplar-based approach with solutions to these problems. An implementation of our approach, called Protos, has been applied to the domain of clinical audiology. After reasonable training, Protos achieved a competence level equaling that of human experts and far surpassing that of other machine learning programs. Additionally, an "ablation study" has identified the aspects of Protos that are primarily responsible for its success. 1 Introduction This paper describes a successful approach to the task of concept learning for heuristic clas...

Abstract. Open Mind Common Sense is a knowledge acquisition system designed to acquire commonsense knowledge from the general public over the web. We describe and evaluate our first fielded system, which enabled the construction of a 450,000 assertion commonsense knowledge base. We then discuss how our second-generation system addresses weaknesses discovered in the first. The new system acquires facts, descriptions, and stories by allowing participants to construct and fill in natural language templates. It employs word-sense disambiguation and methods of clarifying entered knowledge, analogical inference to provide feedback, and allows participants to validate knowledge and in turn each other. 1

CATO is an intelligent learning environment designed to help beginning law students learn basic skills of making arguments with cases. Using CATO, students practice tasks of induction and analogical argumentation. They practice testing theories against a body of cases and making written arguments about a problem, comparing and contrasting it to past cases. CATO’s model addresses arguments in which two opponents analogize a problem to favorable cases, distinguish unfavorable cases, assess the significance of similarities and differences between cases in light of normative knowledge about the domain, and use that knowledge to organize multi-case arguments. CATO communicates the model to students by presenting dynamically-generated argumentation examples and by reifying argument structure based on the model. CATO also provides a case database and tools based on the model that help make students ’ tasks more manageable. CATO was evaluated in the context of an actual legal writing course, in a study involving 30 first-year law students. We found that instruction with CATO leads to statistically significant improvement in students ’ basic argumentation skills, comparable

Developing tools that allow non-programmers to enter knowledge has been an ongoing challenge for AI. In recent years researchers have investigated a variety of promising approaches to knowledge acquisition (KA), but they have often been driven by the needs of knowledge engineers rather than by end users. This paper reports on a series of experiments that we conducted in order to understandhow far a particular KA tool that we are developing is from meeting the needs of end users, and to collect valuable feedback to motivate our future research. This KA tool, called EMeD, exploits Interdependency Models that relate individual components of the knowledge base in order to guide users in specifying problem-solving knowledge. We describe how our experiments helped us addressseveral questions and hypotheses regarding the acquisition of problem-solving knowledge from end users and the benefits of Interdependency Models, and discuss what we learned in terms of improving not only...

The availability of on-line corpora is rapidly changing the field of natural language processing (NLP) from one dominated by theoretical models of often very specific linguistic phenomena to one guided by computational models that simultaneously account for a wide variety of phenomena that occur in real-world text. Thus far, among the best-performing and most robust systems for reading and summarizing large amounts of real-world text are knowledge-based natural language systems. These systems rely heavily on domain-specific, handcrafted knowledge to handle the myriad syntactic, semantic, and pragmatic ambiguities that pervade virtually all aspects of sentence analysis. Not surprisingly, however, generating this knowledge for new domain...

This paper describes a series of experiments with a range of users to evaluate an intelligent interface for acquiring problem-solving knowledge to describe how to accomplish a task. The tool derives the interdependencies between different pieces of knowledge in the system and uses them to guide the user in completing the acquisition task. The paper describes results obtained when the tool was tested with a wide range of users, including end users. The studies show that our acquisition interface saves users an average of 32 % of the time it takes to add new knowledge, and highlight some interesting differences across user groups. The paper also describes what are the areas that need to be addressed in future research in order to make these tools usable by end users.

This paper is concerned with the task of incrementally acquiring and refining the knowledge and algorithms of a knowledge-based system in order to improve its performance over time. In particular, we present the design of DE-KART, a tool whose goal is to provide increasing levels of assistance in acquiring and refining indexing and retrieval knowledge for a knowledge-based retrieval system. DE-KART starts with knowledge that has been entered manually, and increases its level of assistance in acquiring and refining that knowledge, both in terms of the increased level of automation in interacting with users, and in terms of the increased generality of the knowledge. DE-KART is at the intersection of machine learning and knowledge acquisition: it is a first step towards a system which moves along a continuum from interactive knowledge acquisition to increasingly automated machine learning as it acquires more knowledge and experience.

Conceptual information retrieval systems use structured document indices, domain knowledge and a set of heuristic retrieval strategies to match user queries with a set of indices describing the document's content. Such retrieval strategies increase the set of relevant documents retrieved (increase recall), but at the expense of returning additional irrelevant documents (decrease precision). Usually in conceptual information retrieval systems this tradeoff is managed by hand and with difficulty. This paper discusses ways of managing this tradeoff by the application of standard induction algorithms to refine the retrieval strategies in an engineering design domain. We gathered examples of query/retrieval pairs during the system's operation using feedback from a user on the retrieved information. We then fed these examples to the induction algorithm and generated decision trees that refine the existing set of retrieval strategies. We found that (1) induction improved the precision on a set of queries generated by another user, without a significant loss in recall, and (2) in an interactive mode, the decision trees pointed out flaws in the retrieval and indexing knowledge and suggested ways to refine the retrieval strategies. 1.

Traditional approaches to developing user models, especially for computer-based learning environments, are notoriously difficult and time-consuming because they rely heavily on expert-elicited knowledge about the target application and domain. Furthermore, because the expert-elicited knowledge used in the user model is application and domain specific, the entire model development process must be repeated for each new application. In this thesis, we outline a data-based user modeling framework that uses both unsupervised and supervised machine learning in order to reduce the development costs of building user models, and facilitate transferability. We apply the framework to build user models of student interaction with two different learning environments (the CIspace Constraint Satisfaction Problem Applet for demonstrating an Artificial Intelligence algorithm, and the Adaptive Coach for Exploration for mathematical functions), and using two different data sources (logged interface and eye-tracking data). Although these two experiments are limited by the fact that we do not have large data sets, our results provide initial evidence that (i) the framework can automatically identify meaningful student interaction behaviors, and (ii) the user models built via the framework can recognize new student behaviors online. In addition, the similar results obtained from both of our experiments show framework transferability across applications and data types. iii