This article describes an approach to combining symbolic and connectionist approaches to machine learning. A three-stage framework is presented and the research of several groups is reviewed with respect to this framework. The first stage involves the insertion of symbolic knowledge into neural networks, the second addresses the refinement of this prior knowledge in its neural representation, while the third concerns the extraction of the refined symbolic knowledge. Experimental results and open research issues are discussed.

An algorithm that learns from a set of examples should ideally be able to exploit the available resources of (a) abundant computing power and (b) domain-specific knowledge to improve its ability to generalize. Connectionist theory-refinement systems, which use background knowledge to select a neural network's topology and initial weights, have proven to be effective at exploiting domain-specific knowledge; however, most do not exploit available computing power. This weakness occurs because they lack the ability to refine the topology of the neural networks they produce, thereby limiting generalization, especially when given impoverished domain theories. We present the REGENT algorithm which uses (a) domain-specific knowledge to help create an initial population of knowledge-based neural networks and (b) genetic operators of crossover and mutation (specifically designed for knowledge-based networks) to continually search for better network topologies. Experiments on three real-world domains indicate that our new algorithm is able to significantly increase generalization compared to a standard connectionist theory-refinement system, as well as our previous algorithm for growing knowledge-based networks.

Traditional connectionist theory-refinement systems map the dependencies of a domainspecific rule base into a neural network, and then refine this network using neural learning techniques. Most of these systems, however, lack the ability to refine their network's topology and are thus unable to add new rules to the (reformulated) rule base. Therefore, on domain theories that are lacking rules, generalization is poor, and training can corrupt the original rules, even those that were initially correct. We present TopGen, an extension to the Kbann algorithm, that heuristically searches for possible expansions to Kbann's network. TopGen does this by dynamically adding hidden nodes to the neural representation of the domain theory, in a manner analogous to adding rules and conjuncts to the symbolic rule base. Experiments indicate that our method is able to heuristically find effective places to add nodes to the knowledge bases of four realworld problems, as well as an artificial chess domai...

Many scientific and industrial problems can be better understood by learning from samples of the task at hand. For this reason, the machine learning and statistics communities devote considerable research effort on generating inductive-learning algorithms that try to learn the true "concept" of a task from a set of its examples. Often times, however, one has additional resources readily available, but largely unused, that can improve the concept that these learning algorithms generate. These resources include available computer cycles, as well as prior knowledge describing what is currently known about the domain. Effective utilization of available computer time is important since for most domains an expert is willing to wait for weeks, or even months, if a learning system can produce an improved concept. Using prior knowledge is important since it can contain information not present in the current set of training examples. In this thesis, I present three "anytime" approaches to connec...

Local Model Networks are flexible architectures for the representation of complex non-linear dynamic systems. The local nature of the representation leads to a modular network which can integrate a variety of paradigms (neural nets, statistics, fuzzy systems and a priori mathematical models), but because of the power of the local models, the architecture is less sensitive to the curse of dimensionality than other local representations, such as Radial Basis Function networks. The concept of `locality' is a difficult one to define, and tends to vary over a problem's input space, so a constructive structure identification algorithm is presented which automatically defines a suitable model structure on the basis of the observed data from the process being identified. Local learning algorithms are introduced for the local model parameter optimisation, which save computational effort and produce more interpretable and robust models. 1. Introduction Computationally intensive learning systems...

There is great interest in understanding the intrinsic knowledge neural networks have acquired during training. Most work in this direction is focussed on the multi-layer perceptron architecture. The topic of this paper is networks of Gaussian basis functions which are used extensively as learning systems in neural computation. We show that networks of Gaussian basis functions can be generated from simple probabilistic rules. Also, if appropriate learning rules are used, probabilistic rules can be extracted from trained networks. We present methods for the reduction of network complexity with the goal of obtaining concise and meaningful rules. We show how prior knowledge can be refined or supplemented using data by employing either a Bayesian approach, by a weighted combination of knowledge bases, or by generating artificial training data representing the prior knowledge. We validate our approach using a standard statistical data set.

Focusing on examples of knowledge systems and machine learning, this paper illustrates the transfer of AI technology from science to real-world applications. Decades of AI research precede a rather short but significant period, in which companies report the useful exploitation of AI technology. This paper illustrates the role played by science, and it argues that AI is just beginning to produce an ever increasingly variety of real world applications. Keyword Codes: I.2.0; I.2.1; I.2.6 Keywords: Artificial Intelligence, General; Applications and Expert Systems; Learning 1. Introduction Over the last 30 years, research on Artificial Intelligence (AI) has produced a rich variety of techniques for the acquisition, representation and processing of knowledge. In the early years, research on AI was centered around human intelligence, in particular reasoning and cognition. Today, AI has become a rather broad field, ranging from expert systems and theorem provers to evolutionary algorithms, f...

We demonstrate in this paper how certain forms of rule-based knowledge can be used to prestructure a neural network of normalized basis functions and give a probabilistic interpretation of the network architecture. We describe several ways to assure that rule-based knowledge is preserved during training and present a method for complexity reduction that tries to minimize the number of rules and the number of conjuncts. After training, the refined rules are extracted and analyzed.

A consistent pattern of changes in the 31 P MR spectra of normal premenopausal breast during the menstrual cycle has been observed. These encouraging preliminary data suggest that magnetic resonance spectroscopy (MRS) may have a role in monitoring hormone-dependent...

A rolling mill process control system calculates the setup for the mill’s actuators based on models of the technological process. Neural networks are applied as components of hybrid neuro/analytical process models. They are the key to fit the general physical models to the needs of the automation of a specific mill. Present applications include the calculation of the rolling force and strip temperature (hot and cold rolling); prediction of width-spread in the finishing mill; control of strip width shape; and control of the coiling temperature in a cooling train (hot rolling). The authors outline how significant benefits are achieved in rolling mill technology by using neural networks. The work presented here is the result of a close