The most predominant language processing theories have, for some time, been based largely on structured knowledge and relatively simple rules. These symbolic models intentionally segregate syntactic information processing from statistical information as well as semantic, pragmatic, and discourse influences, thereby minimizing the importance of these potential constraints in learning and processing language. While such models have the advantage of being relatively simple and explicit, they are inadequate to account for learning and validated ambiguity resolution phenomena. In recent years, interactive constraint-based theories of sentence processing have gained increasing support, as a growing body of empirical evidence demonstrates early influences of various factors on comprehension performance. Connectionist networks are one form of model that naturally reflect many properties of constraint-based theories, and thus provide a form in which those theories may be instantiated. Unfortunately, most of the connectionist language models implemented until now have involved severe limitations, restricting the phenomena they could address. Comprehension and production models have, by and large, been limited to simple sentences with small vocabularies (cf. St. John & McClelland, 1990). Most models that have addressed the problem of complex, multi-clausal sentence processing have been prediction networks (cf. Elman, 1991; Christiansen & Chater, 1999a). Although a useful component of a language processing system, prediction does not get at the heart of language: the interface between syntax and semantics.

This article explores the use of Simple Synchrony Networks (SSNs) for learning to parse English sentences drawn from a corpus of naturally occurring text. Parsing natural language sentences requires taking a sequence of words and outputting a hierarchical structure representing how those words fit together to form constituents. Feed-forward and Simple Recurrent Networks have had great difficulty with this task, in part because the number of relationships required to specify a structure is too large for the number of unit outputs they have available. SSNs have the representational power to output the necessary O(n 2 ) possible structural relationships, because SSNs extend the O(n) incremental outputs of Simple Recurrent Networks with the O(n) entity outputs provided by Temporal Synchrony Variable Binding. This article presents an incremental representation of constituent structures which allows SSNs to make effective use of both these dimensions. Experiments on learning to ...

We present a connectionist architecture and demonstrate that it can learn syntactic parsing from a corpus of parsed text. The architecture can represent syntactic constituents, and can learn generalizations over syntactic constituents, thereby addressing the sparse data problems of previous connectionist architectures. We apply these Simple Synchrony Networks to mapping sequences of word tags to parse trees. After training on parsed samples of the Brown Corpus, the networks achieve precision and recall on constituents that approaches that of statistical methods for this task. 1 Introduction Connectionist networks are popular for many of the same reasons as statistical techniques. They are robust and have effective learning algorithms. They also have the advantage of learning their own internal representations, so they are less constrained by the way the system designer formulates the problem. These properties and their prevalence in cognitive modeling has generated significant intere...

The Simple Synchrony Network (SSN) is a new connectionist architecture, incorporating the insights of Temporal Synchrony Variable Binding (TSVB) into Simple Recurrent Networks. The use of TSVB means SSNs can output representations of structures, and can learn generalisations over the constituents of these structures (as required by systematicity). This paper describes the SSN and an associated training algorithm, and demonstrates SSNs' generalisation abilities through results from training SSNs to parse real natural language sentences. 1 Introduction Temporal Synchrony Variable Binding (TSVB) [1] extends the representational ability of a connectionist network to include entities. The original motivation behind TSVB was for a network to represent variables and so carry out symbolic reasoning [1]. Henderson [2] argues that this extension further gives connectionist networks an inherent ability to learn generalisations across entities. This ability allows TSVB networks to learn the kinds...

this reportedly improved the learning. And still another important contribution in this work was a method for representing recursive structures -- by means of symbolic transformation of any tree structure into a binary tree, which can easily be transformed to a sequence. Those two operations are reversible,

Traditional approaches to language processing have been based on explicit, discrete representations which are difficult to learn from a reasonable linguistic environment—hence, it has come to be accepted that much of our linguistic representations and knowledge is innate. With its focus on learning based upon graded, malleable, distributed representations, connectionist modeling has reopened the question of what could be learned from the environment in the absence of detailed innate knowledge. This paper provides an overview of connectionist models of language processing, at both the lexical and sentence levels.

this article are of varying nature: position summaries, individual research summaries, historical accounts, discussion of controversial issues, etc. We have not attempted to connect the various pieces together, or to organize them within a coherent framework. Despite this, we think, the reader will find this collection useful.

structure onto these constituents, as would a PDA. At the symbolic level of abstraction, TSVB networks are equivalent to MAs. This result allows us to compare the computational power of TSVB networks to that of well tested symbolic computational models, in particular the PDA (which generates context free languages) . For psycholinguistic models, the computational power of an MA compares favorably to that of a PDA because of the way it specifies word order constraints and other relationships between constituents. Temporal Synchrony Variable Binding is an extension to recurrent networks. Like standard recurrent networks, TSVB networks take a sequence of inputs, perform a sequence of computations, and produce a sequence of outputs. This allows the ordering of the words in a sentence to be represented using the order in which the words are presented to the network. In other words, it allows time to be used to represent the sequence positions of words in the sentence. In addition to this u

this article are of varying nature: position summaries, individual research summaries, historical accounts, discussion of controversial issues, etc. No attempt was made to connect up the various pieces, nor to organize them in a coherent order. Despite this, we think the reader will find this collection useful.

We present a connectionist architecture and demon- strate that it cn learn syntactic parsing from a corpus of parsed text. The architecture can represent syntactic constituents, and can learn generalizations over syntactic constituents, thereby addressing the sparse data problems of previous connectionist chitectures. We apply these Simple Synchrony Networks to mapping sequences of word tags to parse trees. After training on parsed samples of the Brown Corpus, the networks achieve precision and recall on constituents that approaches that of statistical methods for this task.