Recent developments in connectionist architectures for symbolic computation have made it possible to investigate parsing in a connectionist network while still taking advantage of the large body of work on parsing in symbolic frameworks. The work discussed here investigates syntactic parsing in the temporal synchrony variable binding model of symbolic computation in a connectionist network. This computational architecture solves the basic problem with previous connectionist architectures, while keeping their advantages. However, the architecture does have some limitations, which impose constraints on parsing in this architecture. Despite these constraints, the architecture is computationally adequate for syntactic parsing. In addition, the constraints make some significant linguistic predictions. These arguments are made using a specific parsing model. The extensive use of partial descriptions of phrase structure trees is crucial to the ability of this model to recover the syntactic st...

Our ability to see a particular object or situation in one context as being “the same as” another object or situation in another context is the essence of analogy-making. It encompasses our ability to explain new concepts in terms of already-familiar ones, to emphasize particular aspects of situations, to generalize, to characterize situations, to explain

This paper describes techniques for integrating neural networks and symbolic components into powerful hybrid systems. Neural networks have unique processing characteristics that enable tasks to be performed that would be di cult or intractable for a symbolic rule-based system. However, a stand-alone neural network requires an interpretation either by ahuman or a rulebased system. This motivates the integration of neural/symbolic techniques within a hybrid system. Anumber of integration possibilities exist: some systems consist of neural network components performing symbolic tasks while other systems are composed of several neural networks and symbolic components, each component acting as a self-contained module communicating with the others. Other hybrid systems are able to transform subsymbolic representations into symbolic ones and vice-versa. This paper providesanoverview and evaluation of the state of the artofseveral hybrid neural systems for rule-based processing. 1

Psycholinguistic research has shown that the influence of abstract syntactic knowledge on performance is shaped by particular sentences that have been experienced. To explore this idea, the authors applied a connectionist model of sentence production to the development and use of abstract syntax. The model makes use of (a) error-based learning to acquire and adapt sequencing mechanisms and (b) meaning–form mappings to derive syntactic representations. The model is able to account for most of what is known about structural priming in adult speakers, as well as key findings in preferential looking and elicited production studies of language acquisition. The model suggests how abstract knowledge and concrete experience are balanced in the development and use of syntax.

Biological networks are capable of gradual learning based on observing a large number of exemplars over time as well as of rapidly memorizing specific events as a result of a single exposure. The focus of research in neural networks has been on gradual learning, and the modeling of one-shot memorization has received relatively little attention. Nevertheless, the development of biologically plausible computational models of rapid memorization is of considerable value, since such models would enhance our understanding of the neural processes underlying episodic memory formation. A few researchers have attempted the computational modeling of rapid (one-shot) learning within a framework described variably as recruitment learning and vicinal algorithms. Here it is shown that recruitment learning and vicinal algorithms can be grounded in the biological phenomena of long-term potentiation and longterm depression. Toward this end, a computational abstraction of LTP and LTD is presented, and an "algorithm" for the recruitment of binding-detector (or coincidence-detector) cells is described and evaluated using biologically realistic data.

Human cognition is unique in the way in which it relies on combinatorial (or compositional) structures. Language provides ample evidence for the existence of combinatorial structures, but they can also be found in visual cognition. To understand the neural basis of human cognition, it is therefore essential to understand how combinatorial structures can be instantiated in neural terms. In his recent book on the foundations of language, Jackendoff formulated four fundamental problems for a neural instantiation of combinatorial structures: the massiveness of the binding problem, the problem of 2, the problem of variables and the transformation of combinatorial structures from working memory to long-term memory. This paper aims to show that these problems can be solved by means of neural ‘blackboard ’ architectures. For this purpose, a neural blackboard architecture for sentence structure is presented. In this architecture, neural structures that encode for words are temporarily bound in a manner that preserves the structure of the sentence. It is shown that the architecture solves the four problems presented by Jackendoff. The ability of the architecture to instantiate sentence structures is illustrated with examples of sentence complexity observed in human language performance. Similarities exist between the architecture for sentence structure and blackboard architectures for combinatorial structures in visual cognition, derived from the structure of the visual cortex. These architectures are briefly discussed, together with an example of a combinatorial structure in which the blackboard architectures for language and vision are combined. In this way, the architecture for language is grounded in perception. 2 Content

This article explores the use of Simple Synchrony Networks (SSNs) for learning to parse English sentences drawn from a corpus of naturally occurring text. Parsing natural language sentences requires taking a sequence of words and outputting a hierarchical structure representing how those words fit together to form constituents. Feed-forward and Simple Recurrent Networks have had great difficulty with this task, in part because the number of relationships required to specify a structure is too large for the number of unit outputs they have available. SSNs have the representational power to output the necessary O(n 2 ) possible structural relationships, because SSNs extend the O(n) incremental outputs of Simple Recurrent Networks with the O(n) entity outputs provided by Temporal Synchrony Variable Binding. This article presents an incremental representation of constituent structures which allows SSNs to make effective use of both these dimensions. Experiments on learning to ...

Constructivism is the Piagetian notion that learning leads the child to develop new types of representations. For example, on the Piagetian view, a child is born without knowing that objects persist in time even when they are occluded; through a process of learning, the child comes to know that objects persist in time. The trouble with this view has always been the lack of a concrete, computational account of how a learning mechanism could lead to such a change. Recently, however, in a book entitled Rethinking Innateness, Elman et al. (Elman,

The ability to combine words into novel sentences has been used to argue that humans have symbolic language production abilities. Critiques of connectionist models of language often center on the inability of these models to generalize symbolically (Fodor & Pylyshyn, 1988; Marcus, 1998). To address these issues, a connectionist model of sentence production was developed. The model had variables (role-concept bindings) that were inspired by spatial representations (Landau & Jackendoff, 1993). In order to take advantage of these variables, a novel dual-pathway architecture with event semantics is proposed and shown to be better at symbolic generalization than several variants. This architecture has one pathway for mapping message content to words and a separate pathway that enforces sequencing constraints. Analysis of the model’s hidden units demonstrated that the model learned different types of information in each pathway, and that the model’s compositional behavior arose from the combination of these two pathways. The model’s ability to balance symbolic and statistical behavior in syntax acquisition and to model aphasic double dissociations provided independent support for the dual-pathway architecture.

xi 1 Introduction 1 1.1 Intuitions and Evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Previous Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2.1 The Classical Dual Route Model . . . . . . . . . . . . . . . . . . . . . . . 6 1.2.2 Seidenberg and McClelland 1989 . . . . . . . . . . . . . . . . . . . . . . 10 1.2.3 Plaut and Shallice 1993 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2.4 Plaut et al. 1996: Naming . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2.5 Bullinaria 1996 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.2.6 Plaut 1997: Lexical Decision . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.2.7 Harm and Seidenberg 1998: Naming . . . . . . . . . . . . . . . . . . . . 16 1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2 A New Computational Model 18 2.1 Principles . . . . . . . . . . . . . . . . . . . . . . . . ...