Neural networks, despite their empirically-proven abilities, have been little used for the refinement of existing knowledge because this task requires a three-step process. First, knowledge in some form must be inserted into a neural network. Second, the network must be refined. Third, knowledge must be extracted from the network. We have previously described a method for the first step of this process. Standard neural learning techniques can accomplish the second step. In this paper, we propose and empirically evaluate a method for the final, and possibly most difficult, step. This method efficiently extracts symbolic rules from trained neural networks. The four major results of empirical tests of this method are that the extracted rules: (1) closely reproduce (and can even exceed) the accuracy of the network from which they are extracted; (2) are superior to the rules produced by methods that directly refine symbolic rules; (3) are superior to those produced by previous techniques fo...

This paper uses a special notation for specifying locations in a DNA sequence. The idea is to number locations with respect to a fixed, biologically-meaningful, reference point. Negative numbers indicate sites preceding the reference point (by biological convention, this appears on the left) while positive numbers indicate sites following the reference point. (Zero is not used.) Figure 9 illustrates this numbering scheme. Rules use this referencing scheme by stating a position with respect to the reference location, denoted by `@', and the giving a subsequence in the positive direction. For example @-4`GGT' refers to the three nucleotide long sequence in Figure 9 that begins at position-4 and ends at position-2. In addition to this notation for specifying locations a DNA sequence, Table 3 specifies a standard coding scheme for referring to any possible combination of nucleotides using a single letter (IUB Nomenclature Committee, 1985). This scheme is compatible with the codes used by the EMBL, GenBank, and PIR data libraries, three major collections of data for molecular biology. 4.2 Promoter recognition

Many scientific and industrial problems can be better understood by learning from samples of the task at hand. For this reason, the machine learning and statistics communities devote considerable research effort on generating inductive-learning algorithms that try to learn the true "concept" of a task from a set of its examples. Often times, however, one has additional resources readily available, but largely unused, that can improve the concept that these learning algorithms generate. These resources include available computer cycles, as well as prior knowledge describing what is currently known about the domain. Effective utilization of available computer time is important since for most domains an expert is willing to wait for weeks, or even months, if a learning system can produce an improved concept. Using prior knowledge is important since it can contain information not present in the current set of training examples. In this thesis, I present three "anytime" approaches to connec...

Artificial neural networks have proven to be a successful, general method for inductive learning from examples. However, they have not often been viewed in terms of constructive induction. We describe a method for using a knowledgebased neural network of the kind created by the Kbann algorithm as the basis of a system for constructive induction. After training, we extract two types of rules from a network: modified versions of the rules initially provided to the knowledgebased neural network, and rules which describe newly constructed features. Our experiments show that the extracted rules are more accurate, at classifying novel examples, than the trained network from which the rules are extracted. 1 INTRODUCTION Artificial neural networks (ANNs) have proven to be a powerful and general technique for machine learning. For example, a host of empirical comparisons indicate that ANNs are at least as effective at generalizing from training to testing examples as any of several common sym...

The usage of ANNs in "safety-critical" domains, which include the economic and financial applications, is hindered by their "black box"- type approach, which makes it difficult to verify and debug software that includes ANN components. Significant advantages can be gained by combining the symbolic knowledge of a domain theory (DT), with the empirical sub-symbolic knowledge stored in an ANN trained on examples. Rule extraction adds the needed explanation/comprehension component to the much prized ability of ANN to generalise over a learned set of examples. Compiling rules into the an ANN provides better initial conditions for training the network and can significantly improve the speed of learning. The mixed approach allows building hybrid systems that co-operatively combine ANN and AI techniques, increasing both robustness and flexibility. The paper gives an overview of the bases of ANN knowledge extraction under the form of logical functions. ANN design relations are established. 1.

Knowledge-based neural networks are networks whose topology is determined by mapping the dependencies of a domain-specific rulebase into a neural network. However, existing network training methods lack the ability to add new rules to the (reformulated) rulebases. Thus, on domain theories that are lacking rules, generalization is poor, and training can corrupt the original rules, even those that were initially correct. We present TopGen, an extension to the Kbann algorithm, that heuristically searches for possible expansions of a knowledge-based neural network, guided by the domain theory, the network, and the training data. TopGen does this by dynamically adding hidden nodes to the neural representation of the domain theory, in a manner analogous to adding rules and conjuncts to the symbolic rule base. Experiments indicate that our method is able to heuristically find effective places to add nodes to the knowledge bases of four real-world problems, as well as an artificial chess domai...