This paper describes techniques for integrating neural networks and symbolic components into powerful hybrid systems. Neural networks have unique processing characteristics that enable tasks to be performed that would be di cult or intractable for a symbolic rule-based system. However, a stand-alone neural network requires an interpretation either by ahuman or a rulebased system. This motivates the integration of neural/symbolic techniques within a hybrid system. Anumber of integration possibilities exist: some systems consist of neural network components performing symbolic tasks while other systems are composed of several neural networks and symbolic components, each component acting as a self-contained module communicating with the others. Other hybrid systems are able to transform subsymbolic representations into symbolic ones and vice-versa. This paper providesanoverview and evaluation of the state of the artofseveral hybrid neural systems for rule-based processing. 1

Massively parallel networks of relatively simple computing elements offer an attractive and versatile framework for exploring a variety of learning structures and processes for intelligent systems. This paper briefly summarizes the popular learning structures and processes used in such networks. It outlines a range of potentially more powerful alternatives for pattern-directed inductive learning in such systems. It motivates and develops a class of new learning algorithms for massively parallel networks of simple computing elements. We call this class of learning processes generative for they offer a set of mechanisms for constructive and adaptive determination of the network architecture - the number of processing elements and the connectivity among them - as a function of experience. Generative learning algorithms attempt to overcome some of the limitations of some approaches to learning in networks that rely on modification of weights on the links within an otherwise fixed network t...

This memory can take several forms based on the time scales at which such modifications are allowed. Some symbol structures might have the property of determining choice and the order of application of transformations to be applied on other symbol structures. These are essentially the programs. Programs when executed --- typically through the conventional process of compilation and interpretation and eventually --- when they operate on symbols that are linked through grounding to particular effectors --- produce behavior. Working memory holds symbol structures as they are being processed. Long-term memory, generally speaking, is the repository of programs and can be changed by addition, deletion, or modification of symbol structures that it holds. Such a system can compute any Turing-computable function provided it has sufficiently large memory and its primitive set of transformations are adequate for the composition of arbitrarily symbol structures (programs) and the interpreter is capable of interpreting any possible symbol structure. This also means that any particular set of symbolic processes can be carried out by an NANN --- provided it has potentially infinite memory, or finds a way to use its transducers and effectors to use the external physical environment to serve as its memory). 14 Chapter 12 Knowledge in SAI systems is typically embedded in complex symbol structures such as lists (Norvig, 1992), logical databases (Genesereth and Nilsson, 1987), semantic networks (Quillian, 1968), frames (Minsky, 1975), schemas (Arbib, 1972; 1994), and manipulated by (often serial) procedures or inferences (e.g., list processing, application of production rules (Waterman, 1985), or execution of logic programs (Kowalski, 1977) carried out by a central processor that accesse...

Artificial neural networks (ANN's), due to their inherent parallelism, offer an attractive paradigm for implementation of symbol processing systems for applications in computer science and artificial intelligence. This paper explores systematic synthesis of modular neural-network architectures for syntax analysis using a prespecified grammar---a prototypical symbol processing task which finds applications in programming language interpretation, syntax analysis of symbolic expressions, and high-performance compilers. The proposed architecture is assembled from ANN components for lexical analysis, stack, parsing and parse tree construction. Each of these modules takes advantage of parallel content-based pattern matching using a neural associative memory. The proposed neural-network architecture for syntax analysis provides a relatively efficient and high performance alternative to current computer systems for applications that involve parsing of LR grammars which constitute a widely used subset of deterministic context-free grammars. Comparison of quantitatively estimated performance of such a system [implemented using current CMOS very large scale integration (VLSI) technology] with that of conventional computers demonstrates the benefits of massively parallel neuralnetwork architectures for symbol processing applications.

The system HTRP -- Hybrid Thematic Role Processor -- is a symbolic-connectionist hybrid system, combining the advantages of symbolic approaches with the advantages of connectionism, in order to process the thematic roles, the semantic relations between words in a sentence. However, HTRP has some limitations: the sentences must be broken into verb-noun pairs to be presented to the network. This makes it impossible for the system to deal with instances in which constraints are operative not only between the verb and one of its arguments (nouns), but also between two arguments of the same verb. Another possible dra wback is training with negative examples (semantically unsound sentences). Although many researchers point out that negative inputs are necessary for a system to learn a grammar, several authors believe that, under certain circumstances, a network is able to learn in absence of negative examples. From a psycholinguistic standpoint, especially regarding language acquisition, explicit negative evidence is hardly to be expected as part of the cognitive environment. In this paper, new versions of HTRP are proposed (HTRP II) to account for the whole sentence as input with no negative examples provided during training.

This memory can take several forms based on the time scales at which such modifications are allowed. Some symbol structures might have the property of determining choice and the order of application of transformations to be applied on other symbol structures. These are essentially the programs. Programs when executed -- typically through the conventional process of compilation and interpretation and eventually -- when they operate on symbols that are linked through grounding to particular effectors -- produce behavior. Working memory holds symbol structures as they are being processed. Long--term memory, generally speaking, is the repository of programs and can be changed by addition, deletion, or modification of symbol structures that it holds. The reader is refered to (Newell, 1990) for a detailed treatment of symbol systems of this sort. Such a symbol system can compute any Turing--computable function provided it has sufficiently large memory and its primitive set of transformations are Beyond Symbolic AI and Connectionist Networks 7 adequate for the composition of arbitrarily symbol structures (programs) and the interpreter is capable of interpreting any possible symbol structure. This also means that any particular set of symbolic processes can be carried out by a CN -- provided it has potentially infinite memory, or finds a way to use its transducers and effectors to use the external physical environment to augment its memory (just as humans have in their use of stone tablets, papyrus, and books through the ages). Knowledge in SAI systems is typically embedded in complex symbol structures such as lists (Norvig, 1992), logical databases (Genesereth and Nilsson, 1987), semantic networks (Quillian, 1968), frames (Minsky, 1975), schemas (Arbib, 1972; 1994), and mani...

Abstract not found