Associative memories are conventionally used to represent data with very simple structure: sets of pairs of vectors. This paper describes a method for representing more complex compositional structure in distributed representations. The method uses circular convolution to associate items, which are represented by vectors. Arbitrary variable bindings, short sequences of various lengths, simple framelike structures, and reduced representations can be represented in a fixed width vector. These representations are items in their own right, and can be used in constructing compositional structures. The noisy reconstructions extracted from convolution memories can be cleaned up by using a separate associative memory that has good reconstructive properties.

A number of connectionist models capable of representing data with compositional structure have recently appeared. These new models suggest the intriguing possibility of performing holistic structure-sensitive computations with distributed representations. Two possible forms of holistic inference, transformational inference and confluent inference, are identified and compared. Transformational inference was successfully demonstrated in [Chalmers, 1990]; however, since the pure transformational approach does not consider the eventual inference tasks during the process of learning its representations, there is a drawback that the holistic transformation corresponding to a given inference task could become arbitrarily complex, and thus very difficult to learn. Confluent inference addresses this drawback by achieving a tight coupling between the distributed representations of a problem and the solution for the given inference task while the net is still learning its representations. A dual...

Distributed representations are attractive for a number of reasons. They offer the possibility of representing concepts in a continuous space, they degrade gracefully with noise, and they can be processed in a parallel network of simple processing elements. However, the problem of representing nested structure in distributed representations has been for some time a prominent concern of both proponents and critics of connectionism [Fodor and Pylyshyn 1988; Smolensky 1990; Hinton 1990]. The lack of connectionist representations for complex structure has held back progress in tackling higher-level cognitive tasks such as language understanding and reasoning. In this thesis I review connectionist representations and propose a method for the distributed representation of nested structure, which I call "Holographic Reduced Representations " (HRRs). HRRs provide an implementation of Hinton's [1990] "reduced descriptions". HRRs use circular convolution to associate atomic items, which are rep...

In this paper we propose an extension of the Recursive Auto-Associative Memory (RAAM) by Pollack. This extension, the Labeling RAAM (LRAAM), is able to encode labeled graphs with cycles by representing pointers explicitly. A theoretical analysis of the constraints imposed on the weights by the learning task under the hypothesis of perfect learning and linear output units is presented. Cycles and confluent pointers result to be particularly effective in imposing constraints on the weights. Some technical problems encountered in the RAAM, such as the termination problem in the learning and decoding processes, are solved more naturally in the LRAAM framework. The representations developed for the pointers seem to be robust to recurrent decoding along a cycle. Data encoded in a LRAAM can be accessed by pointer as well as by content. The direct access by content can be achieved by transforming the encoder network of the LRAAM in a Bidirectional Associative Memory (BAM). Different access pro...

Supporters of eliminative connectionism have argued for a pattern association based explanation of language learning and language processing. They deny that explicit rules and symbolic representations play any role in language processing and cognition in general. Their argument is based to a large extent on two artificial neural network (ANN) models that are claimed to be able to learn the past tenses of English verbs. (Rumelhart and McClelland, 1986; MacWhinney and Leinbach, 1991). In this article we critically review Rumelhart and McClelland's as well as MacWhinney and Leinbach's ANN-models and conclude that they do not succeed in the assigned task of learning the past tenses of English verbs. In order to answer their challenge to the symbolic processing approach, we present our Symbolic Pattern Associator (SPA) -- a general purpose pattern associator that can learn to associate arbitrary discrete patterns. We carried out several experiments with the SPA using the same set of verbs ...

It is di cult to clearly de ne the symbolic and subsymbolic paradigms; each is usually described by its tendencies rather than any one de nitive property. Symbolic processing is generally characterized by hard-coded, explicit rules operating on discrete, static tokens, while subsymbolic processing is associated with learned, fuzzy constraints a ecting continuous,

This paper will give an overview of the various approaches to neurosymbolic integration. Roughly, these can be divided into two strategies: unified strategies aim at attaining neural and symbolic capabilities using neural networks alone, while hybrid strategies combine neural networks with symbolic models such as expert systems, case-based reasoning systems, 2 Chapter 2 and decision trees. These two approaches form the main subtrees of the classification hierarchy depicted in Figure 1. Symbol Proc. Neuronal Unified approach Symbol Proc. hybrids Connectionist Localist Hybrid approach Combined L/D Neurosymbolic integration Functional Chainprocessing Translational Subprocessing hybrids Metaprocessing Distributed Coprocessing Figure 1 Classification of integrated neurosymbolic systems.

This paper presents a survey of connectionist inference systems.

Information with structure is traditionally organized into records with fields. For example, a medical record consisting of name, sex, age, and weight might look like (Joe, male, 66, 77). What 77 stands for is determined by its location in the record, so that this is an example of local representation. The brain's wiring, and robustness under local damage, speak for the importance of distributed representations. The Holographic Reduced Representation (HRR) of Plate is a prime example based on real or complex vectors. This paper describes how spatter coding leads to binary HRRs, and how the fields of a record are encoded into a long binary word without fields and how they are extracted from such a word. 1 Introduction Nested compositional structure is fundamental to high-level mental functions, such as language and analogy. Accordingly, modeling these functions with neural nets requires that the structures be represented in a form suitable for neural nets.

Introduction After a long and exhausting war between the representatives of the symbolic and connectionist approaches (this war stimulated, however, the clarification of the limitations and advantages of both approaches) a growing group of peace-makers emerged who tried to integrate the advantages of both approaches and to fill in the gap between them (Hendler, 1989a, Hinton, 1990, Barnden & Pollack, 1991, Thornton, 1991, Sun & Bookman, 1992, 1994, Dinsmore, 1992, Holyoak & Barnden, 1994). However, a mini-war started between the peace-makers themselves on the issue how to sign the peace treaty: with the surrender of one of the approaches or with their parity. Some researchers supported the connectionist-to-the-top view that symbol structures and symbol processing should emerge from the work of a neural network (called a unified approach in chapters 2 and 4 of this volume and connectionist symbol processing in (Pollack, 1990, Smolensky, 1990, Touretzky, 1990, Smolensky et al.,