A major challenge in indexing unstructured hypertext databases is to automatically extract meta-data that enables structured search using topic taxonomies, circumvents keyword ambiguity, and improves the quality of search and profile-based routing and filtering. Therefore, an accurate classifier is an essential component of a hypertext database. Hyperlinks pose new problems not addressed in the extensive text classification literature. Links clearly contain highquality semantic clues that are lost upon a purely termbased classifier, but exploiting link information is non-trivial because it is noisy. Naive use of terms in the link neighborhood of a document can even degrade accuracy. Our contribution is to propose robust statistical models and a relaxation labeling technique for better classification by exploiting link information in a small neighborhood around documents. Our technique also adapts gracefully to the fraction of neighboring documents having known topics. We experimented ...

: FOIL is a learning system that constructs Horn clause programs from examples. This paper summarises the development of FOIL from 1989 up to early 1993 and evaluates its effectiveness on a non-trivial sequence of learning tasks taken from a Prolog programming text. Although many of these tasks are handled reasonably well, the experiment highlights some weaknesses of the current implementation. Areas for further research are identified. 1. Introduction The principal differences between zeroth-order and first-order supervised learning systems are the form of the training data and the way that a learned theory is expressed. Data for zeroth-order learning programs such as ASSISTANT [Cestnik, Kononenko and Bratko, 1986], CART [Breiman, Friedman, Olshen and Stone, 1984], CN2 [Clark and Niblett, 1987] and C4.5 [Quinlan, 1992] comprise preclassified cases, each described by its values for a fixed collection of attributes. These systems develop theories, in the form of decision trees o...

Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset information missing from the other source. By so doing, a hybrid learning system should learn more effectively than systems that use only one of the information sources. KBANN(Knowledge-Based Artificial Neural Networks) is a hybrid learning system built on top of connectionist learning techniques. It maps problem-specific "domain theories", represented in propositional logic, into neural networks and then refines this reformulated knowledge using backpropagation. KBANN is evaluated by extensive empirical tests on two problems from molecular biology. Among other results, these tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several t...

. Learning multiple descriptions for each class in the data has been shown to reduce generalization error but the amount of error reduction varies greatly from domain to domain. This paper presents a novel empirical analysis that helps to understand this variation. Our hypothesis is that the amount of error reduction is linked to the "degree to which the descriptions for a class make errors in a correlated manner." We present a precise and novel definition for this notion and use twenty-nine data sets to show that the amount of observed error reduction is negatively correlated with the degree to which the descriptions make errors in a correlated manner. We empirically show that it is possible to learn descriptions that make less correlated errors in domains in which many ties in the search evaluation measure (e.g. information gain) are experienced during learning. The paper also presents results that help to understand when and why multiple descriptions are a help (irrelevant attribute...

This article describes a comprehensive approach to automatic theory revision. Given an imperfect theory, the approach combines explanation attempts for incorrectly classified examples in order to identify the failing portions of the theory. For each theory fault, correlated subsets of the examples are used to inductively generate a correction. Because the corrections are focused, they tend to preserve the structure of the original theory. Because the system starts with an approximate domain theory, in general fewer training examples are required to attain a given level of performance (classification accuracy) compared to a purely empirical system. The approach applies to classification systems employing a propositional Horn-clause theory. The system has been tested in a variety of application domains, and results are presented for problems in the domains of molecular biology and plant disease diagnosis. 1 INTRODUCTION 2 1 Introduction One of the most difficult problems in the develo...

FOIL is a first-order learning system that uses information in a collection of relations to construct theories expressed in a dialect of Prolog. This paper provides an overview of the principal ideas and methods used in the current version of the system, including two recent additions. We present examples of tasks tackled by FOIL and of systems that adapt and extend its approach. 1. Introduction All symbolic machine learning leads to the formulation or modification of theories, so the language in which theories are expressed is an important consideration. Firstorder theory languages have been used for at least thirty years, as documented by Sammut [1993]. Explanation-based generalisation systems [Mitchell, Keller and Kedar-Cabelli, 1986; DeJong and Mooney, 1986] have always required them, but the early and influential work of Shapiro [1983] and Sammut and Banerji [1986] also employed them in an inductive learning context. Nevertheless, first-order empirical learning, including...

Current theory-guided learning systems are inflexible, in that they are committed to performing one particular class of theory corrections; this is problematic because in some cases special-purpose theory-guided learning systems can dramatically outperform general-purpose ones. To address this problem, we describe a new system in which theory-guided learning is separated into two phases. The first phase is "theory intrepretation ", in which the initial theory is translated into an explicit description of the bias for an inductive learning system; we introduce antecedent description grammars as a language for explicitly representing this bias. The second phase is "grammatically biased learning", in which this bias is used to search for a hypothesis. We demonstrate empirically that this approach leads to a flexible learning system which can, by use of suitable translators, emulate several useful learning systems; we also argue that this architecture makes it easier for a user unfamiliar ...

This chapter describes a multistrategy system that employs independent modules for deductive, abductive, and inductive reasoning to revise an arbitrarily incorrect propositional Horn-clause domain theory to t a set of preclassi ed training instances. By combining such diverse methods, Either is able to handle a wider range of imperfect theories than other theory revision systems while guaranteeing that the revised theory will be consistent with the training data. Either has successfully revised two actual expert theories, one in molecular biology and one in plant pathology. The results con rm the hypothesis that using a multistrategy system to learn from both theory and data gives better results than using either theory or data alone. 1

AI is moving away from "toy tasks" such as block stacking towards real-world problems. This trend is positive, but the amount of preliminary groundwork required to tackle a real-world task can be staggering, particularly when developing an integrated agent architecture. To address this problem, we advicate real-world software environments, such as operating systems or databases, as domains for agent research. The cost, effort, and expertise required to develop and systematically experiment with software agents is relatively low. Furthermore, software environments circumvent many thorny, but peripheral, research issues that are inescapable in other environments. Thus, software environments enable us to test agents ina real world yet focus on core AI research issues. To support this claim, we describe our project to develop UNIX softbots (software robots) -- intelligent agnets that interact with UNIX. Existing softbots accept a diverse set of high-level goals, generate and execute plans to achieve these goals in real time, and recover from errors when necessary.

Learning systems that express theories in firstorder logic must ensure that the theories are executable and, in particular, that they do not lead to infinite recursion. This paper presents a heuristic method for preventing infinite recursion in the (multi-clause) definition of a recursive relation. The method has been implemented in the latest version of foil, but could also be used with any learning method that grows clauses from ground facts by repeated specialization. Results on several examples, including Ackermann's function, are presented. 1 Introduction A growing cohort of systems now learn first-order theories to explain observations. Early examples, mis [Shapiro, 1983] and marvin [Sammut and Banerji, 1986], have been joined by cigol [Muggleton and Buntine, 1988], foil [Quinlan, 1990, 1991], golem [Muggleton and Feng, 1990], focl [Pazzani, Brunk and Silverstein, 1991; Pazzani and Kibler, 1992], ile [Rouveirol, 1991] and others. The reason for looking beyond familiar proposit...