There has been much interest in the possibility of connectionist models whose representations can be endowed with compositional structure, and a variety of such models have been proposed. These models typically use distributed representations that arise from the functional composition of constituent parts. Functional composition and decomposition alone, however, yield only an implementation of classical symbolic theories. This paper explores the possibility of moving beyond implementation by exploiting holistic structure-sensitive operations on distributed representations. An experiment is performed using Pollack’s Recursive Auto-Associative Memory. RAAM is used to construct distributed representations of syntactically structured sentences. A feed-forward network is then trained to operate directly on these representations, modeling syntactic transformations of the represented sentences. Successful training and generalization is obtained, demonstrating that the implicit structure present in these representations can be used for a kind of structure-sensitive processing unique to the connectionist domain. 1

Abstract not found

Abstract not found

Human cognition is unique in the way in which it relies on combinatorial (or compositional) structures. Language provides ample evidence for the existence of combinatorial structures, but they can also be found in visual cognition. To understand the neural basis of human cognition, it is therefore essential to understand how combinatorial structures can be instantiated in neural terms. In his recent book on the foundations of language, Jackendoff formulated four fundamental problems for a neural instantiation of combinatorial structures: the massiveness of the binding problem, the problem of 2, the problem of variables and the transformation of combinatorial structures from working memory to long-term memory. This paper aims to show that these problems can be solved by means of neural ‘blackboard ’ architectures. For this purpose, a neural blackboard architecture for sentence structure is presented. In this architecture, neural structures that encode for words are temporarily bound in a manner that preserves the structure of the sentence. It is shown that the architecture solves the four problems presented by Jackendoff. The ability of the architecture to instantiate sentence structures is illustrated with examples of sentence complexity observed in human language performance. Similarities exist between the architecture for sentence structure and blackboard architectures for combinatorial structures in visual cognition, derived from the structure of the visual cortex. These architectures are briefly discussed, together with an example of a combinatorial structure in which the blackboard architectures for language and vision are combined. In this way, the architecture for language is grounded in perception. 2 Content

xi 1 Introduction 1 1.1 Intuitions and Evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Previous Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2.1 The Classical Dual Route Model . . . . . . . . . . . . . . . . . . . . . . . 6 1.2.2 Seidenberg and McClelland 1989 . . . . . . . . . . . . . . . . . . . . . . 10 1.2.3 Plaut and Shallice 1993 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2.4 Plaut et al. 1996: Naming . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2.5 Bullinaria 1996 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.2.6 Plaut 1997: Lexical Decision . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.2.7 Harm and Seidenberg 1998: Naming . . . . . . . . . . . . . . . . . . . . 16 1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2 A New Computational Model 18 2.1 Principles . . . . . . . . . . . . . . . . . . . . . . . . ...

The main claim of this paper is that connectionism offers cognitive science a number of excellent opportunities for turning methodological, theoretical. and meta-theoretica! schisms into powerfnl integrations--opportunities for forging constructive synergy out of the destructive interference which plagues the field. The paper begins with an analysis of the rifts in tile field and what it would take to overcome them. We argue that while connectionism ha,s often contributed to the deepexLing of these schisms, ]t is nonetheless possible to turn this trend around--possible for connectionism to play a central role in a unification of cognitive science. Essential o this process is the development of strong theoretical principles founded (in part) on connectionist computation; a main goal of this paper is to demonstrate that such principles are indeed within the reach of a connectionist-grounded theory of cognition. The enterprise rests on a willingness to entertain, analyze, and extend characterizations of cognitive problems, and hypothesized solutions, which are deliberately overly simple and general--in order to disco4'er the insights they can offer through mathematical a.na.lyses which this simplicity and generality are makes possible.

In 1988 Fodor and Pylyshyn issued a challenge to the newly-popular connectionism: explain the systematicity of cognition without merely implementing a so-called classical architecture. Since that time quite a number of connectionist models have been put forward, either by their designers or by others, as in some measure demonstrating that the challenge can be met (e.g., Pollack, 1988, 1990; Smolensky, 1990; Chalmers, 1990; Niklasson and Sharkey, 1992; Brousse, 1993). Unfortunately, it has generally been unclear whether these models actually do have this implication (see, for instance, the extensive philosophical debate in Smolensky, 1988; Fodor and McLaughlin, 1990; van Gelder, 1990, 1991; McLaughlin, 1993a, 1993b; Clark, 1993). Indeed, we know of no major supporter of classical orthodoxy who has felt compelled, by connectionist models and arguments, to concede in print that connectionists have in fact delivered a non-classical explanation of systematicity.

Several connectionist models have been supplying non-classical explanations to the challenge of explaining systematicity, i.e., structure sensitive processes, without merely being implementations of classical architectures. However, lately the challenge has been extended to include learning related issues. It has been claimed that when these issues are taken into account, only a restricted form of systematicity could be claimed by the connectionist models put forward so far. In this paper we investigate this issue further, and supply a model and results that satisfies even the revised challenge.

Systematicity, the ability to represent and process structurally related objects, is a significant and pervasive property of cognitive behaviour, and clearly evident in language. In the case of Connectionist models that learn from examples, systematicity is generalization over examples sharing a common structure. Although Connectionist models (e.g., the recurrent network and its variants) have demonstrated generalization over structured domains, there has not beena clear demonstration of strong systematicity (i.e., generalization across syntactic position). The tensor has been proposed as a way of representing structured objects, however, there has not beenan effective learning mechanism (in the strongly systematic sense) to explain how these representations may be acquired. I address this issue through an analysis of tensor learning dynamics. These ideas are then implemented as the tensor-recurrent network which is shown to exhibit strong systematicity on a simple language task. Final...

The problem of representing the spatial structure of images, which arises in visual object processing, is commonly described using terminology borrowed from propositional theories of cognition, notably, the concept of compositionality. The classical propositional stance mandates representations composed of symbols, which stand for atomic or composite entities and enter into arbitrarily nested relationships.