Standard algorithms for explanation-based learning require complete and correct knowledge bases. The KBANN system relaxes this constraint through the use of empirical learning methods to refine approximately correct knowledge. This knowledge is used to determine the structure of an artificial neural network and the weights on its links, thereby making the knowledge accessible for modification by neural learning. KBANN is evaluated by empirical tests in the domain of molecular biology. Networks created by KBANN are shown to be superior, in terms of their ability to correctly classify unseen examples, to randomly initialized neural networks, decision trees, "nearest neighbor" matching, and standard techniques reported in the biological literature. In addition, KBANN's networks improve the initial knowledge in biologically interesting ways. Introduction Explanation-based learning (EBL) (Mitchell et al. 1986; DeJong & Mooney 1986) provides a way of incorporating pre-existing knowledge i...

Abstract Despite the fact that many symbolic and neural network (connectionist) learning algorithms address the same problem of learning from classified examples, very little is known regarding their comparative strengths and weaknesses. Experiments comparing the ID3 symbolic learning algorithm with the perception and backpropagation neural learning algorithms have been performed using five large, real-world data sets. Overall, backpropagation performs slightly better than the other two algorithms in terms of classification accuracy on new examples, but takes much longer to train. Experimental results suggest that backpropagation can work significantly better on data sets containing numerical data. Also analyzed empirically are the effects of (1) the amount of training data, (2) imperfect training examples, and (3) the encoding of the desired outputs. Backpropagation occasionally outperforms the other two systems when given relatively small amounts of training data. It is slightly more accurate than ID3 when examples are noisy or incompletely specified. Finally, backpropagation more effectively utilizes a "distributed " output encoding.

Despite the fact that many symbolic and connectionist (neural net) learning algorithms are addressing the same problem of learning from classified examples, very little Is known regarding their comparative strengths and weaknesses. This paper presents the results of experiments comparing the ID3 symbolic learning algorithm with the perceptron and back-propagation connectionist learning algorithms on several large real-world data sets. The results show that ID3 and perceptron run significantly faster than does backpropagation, both during learning and during classification of novel examples. However, the probability of correctly classifying new examples is about the same for the three systems. On noisy data sets there is some indication that backpropagation classifies more accurately. 1.

Massively parallel networks of relatively simple computing elements offer an attractive and versatile framework for exploring a variety of learning structures and processes for intelligent systems. This paper briefly summarizes the popular learning structures and processes used in such networks. It outlines a range of potentially more powerful alternatives for pattern-directed inductive learning in such systems. It motivates and develops a class of new learning algorithms for massively parallel networks of simple computing elements. We call this class of learning processes generative for they offer a set of mechanisms for constructive and adaptive determination of the network architecture - the number of processing elements and the connectivity among them - as a function of experience. Generative learning algorithms attempt to overcome some of the limitations of some approaches to learning in networks that rely on modification of weights on the links within an otherwise fixed network t...

Abstract not found

Learning when limited to modification of some parameters has a limited scope; the capability to modify the system structure is also needed to get a wider range of the learnable. In the case of artificial neural networks, learning by iterative adjustment of synaptic weights can only succeed if the network designer predefines an appropriate network structure, i.e., number of hidden layers, units, and the size and shape of their receptive and projective fields. This paper advocates the view that the network structure should not, as usually done, be determined by trial-and-error but should be computed by the learning algorithm. Incremental learning algorithms can modify the network structure by addition and/or removal of units and/or links. A survey of current connectionist literature is given on this line of thought. "Grow and Learn" (GAL) is a new algorithm that learns an association at one-shot due to being incremental and using a local representation. During the so-called...

Artificial neural networks have proven to be a successful, general method for inductive learning from examples. However, they have not often been viewed in terms of constructive induction. We describe a method for using a knowledgebased neural network of the kind created by the Kbann algorithm as the basis of a system for constructive induction. After training, we extract two types of rules from a network: modified versions of the rules initially provided to the knowledgebased neural network, and rules which describe newly constructed features. Our experiments show that the extracted rules are more accurate, at classifying novel examples, than the trained network from which the rules are extracted. 1 INTRODUCTION Artificial neural networks (ANNs) have proven to be a powerful and general technique for machine learning. For example, a host of empirical comparisons indicate that ANNs are at least as effective at generalizing from training to testing examples as any of several common sym...

Many authors have suggested that SP (symbol processing) and CN (connectionist network) models offer radically, or even fundamentally, different paradigms for modeling intelligent behavior (see Schneider, 1987) and the design of intelligent systems. Others have argued that CN models have little to contribute to our efforts to understand intelligence (Fodor & Pylyshyn, 1988). A critical examination of the popular characterizations of SP and CN models suggests that neither of these extreme positions is justified. There are many advantages to be gained by a synthesis of the best of both SP and CN approaches in the design of intelligent systems. The Generalized connectionist networks (GCN) (alternately called generalized neuromorphic systems (GNS)) introduced in this paper provide a framework for such a synthesis.

An intelligent system has to be capable of adapting to a constantly changing environment. It therefore, ought to be capable of learning from its perceptual interactions with its surroundings. This requires a certain amount of plasticity in its structure. Any attempt to model the perceptual capabilities of a living system or, for that matter, to construct a synthetic system of comparable abilities, must therefore, account for such plasticity through a variety of developmental and learning mechanisms. This paper examines some results from neuroanatomical, morphological, as well as behavioral studies of the development of visual perception; integrates them into a computational framework; and suggests several interesting experiments with computational models that can yield insights into the development of visual perception. Role of Environmental Experience in Development and Learning In order to understand the development of information processing structures in the brain, one needs knowl...

1 An understanding of learning -- the process by which a learner acquires and refines a broad range of knowledge and skills -- is central to the enterprise of building truly adaptive, flexible, robust, and creative intelligent systems. Significant theoretical and empirical contributions to the characterization of learning in computational terms have emerged from research in a number of disparate research paradigms. The limitations of individual paradigms and of particular classes of techniques within each paradigm are beginning to be recognized. Converging lines of evidence from multiple sources, both theoretical as well as empirical, suggest that artificial intelligence systems, in order to be able to deal with complex tasks such as recognizing and describing 3-dimensional objects, or communicating in natural language, must be able to effectively utilize a range of learning algorithms operating with an adequate repertoire of representational structures. This paper draws on a broad ran...