This paper describes different types of models for the evolution of communication and language. It uses the distinction between signals, symbols, and words for the analysis of evolutionary models of language. In particular, it show how evolutionary computation techniques, such as Artificial Life, can be used to study the emergence of syntax and symbols from simple communication signals. Initially, a computational model that evolves repertoires of isolated signals is presented. This study has simulated the emergence of signals for naming foods in a population of foragers. This type of model studies communication systems based on simple signal-object associations. Subsequently, models that study the emergence of grounded symbols are discussed in general, including a detailed description of a work on the evolution of simple syntactic rules. This model focuses on the emergence of symbol-symbol relationships in evolved languages. Finally, computational models of syntax acquisition and evolution are discussed. These different types of computational models provide an operational def'mition of the signal/symbol/word distinction. The simulation and analysis of these types of models will help understanding the role of symbols and symbol acquisition in the origin of language.

The most predominant language processing theories have, for some time, been based largely on structured knowledge and relatively simple rules. These symbolic models intentionally segregate syntactic information processing from statistical information as well as semantic, pragmatic, and discourse influences, thereby minimizing the importance of these potential constraints in learning and processing language. While such models have the advantage of being relatively simple and explicit, they are inadequate to account for learning and validated ambiguity resolution phenomena. In recent years, interactive constraint-based theories of sentence processing have gained increasing support, as a growing body of empirical evidence demonstrates early influences of various factors on comprehension performance. Connectionist networks are one form of model that naturally reflect many properties of constraint-based theories, and thus provide a form in which those theories may be instantiated. Unfortunately, most of the connectionist language models implemented until now have involved severe limitations, restricting the phenomena they could address. Comprehension and production models have, by and large, been limited to simple sentences with small vocabularies (cf. St. John & McClelland, 1990). Most models that have addressed the problem of complex, multi-clausal sentence processing have been prediction networks (cf. Elman, 1991; Christiansen & Chater, 1999a). Although a useful component of a language processing system, prediction does not get at the heart of language: the interface between syntax and semantics.

Much effort has been expended in the field of Natural Language Understanding in developing methods for deriving the syntactic structure of a text. It is still unclear, however, to what extent syntactic information actually matters for the representation of meaning. LSA (Latent Semantic Analysis) allows you to derive information about the meaning without paying attention even to the order of words within a sentence. This is consistent with the view that syntax plays a subordinate role for semantic processing of text. But LSA does not perform as well as humans do in discriminating meanings. Can syntax

INTRODUCTION Many researchers assume that the most appropriate way to express the systematic aspects of language is in terms of a set of rules. For instance, there is a systematic relationship between the written and spoken forms of most English words (e.g., GAVE f /geIV/), and this relationship can be expressed in terms of a fairly concise set of grapheme-phoneme correspondence (GPC) rules (e.g., G f /g/, A_E f /eI/, V f /v/). In addition to being able to generate accurate pronunciations of so-called regular words, such rules also provide a straightforward account of how skilled readers apply their knowledge to novel items---for Direct all correspondence to: David C. Plaut, Mellon Institute 115-CNBC, Carnegie Mellon University, 4400 Fifth Avenue, Pittsburgh, PA 15213-2683; E-mail: plaut@cmu.edu 543 example, in pronouncing word-like nonwords (e.g., MAVE f /meIV/). Most linguistic domains, however, are only partially systematic. Thus, there are many English words whose pronunciations

Many recent models of language comprehension have stressed the role of distributional frequencies in determining the relative accessibility or ease of processing associated with a particular lexical item or sentence structure. However, there exist relatively few comprehensive analyses of structural frequencies, and little consideration has been given to the appro- priateness of using any particular set of corpus frequencies in modeling human language. We provide a comprehensive set of structural frequencies for a variety of written and spoken corpora, focusing on structures that have played a critical role in debates on normal psycholinguistics, aphasia, and child language acquisition, and compare our results with those from several recent papers to illustrate the implications and limitations of using corpus data in psycholinguistic research.

This article responds to M. C. MacDonald and M. H. Christiansen’s 2002 commentary on the capacity theory of working memory (WM) and its computational implementation, the Capacity-Constrained Collaborative Activation–based Production System (3CAPS). The authors also point out several shortcomings in MacDonald and Christiansen’s proposal for the construal of WM, arguing that at some level of description, their model is a variant of a small subset of the 3CAPS theory. The authors go on to describe how the symbolic and connectionist mechanisms within the hybrid 3CAPS architecture combine to produce a processing style that provides a good match to human sentence comprehension and other types of high-level cognition. The properties of 3CAPS are related to the development of other connectionist, symbolic, and hybrid systems. This article has the goals of (a) refuting some of MacDonald and Christiansen’s (2002) incorrect descriptions of the capacity theory of sentence comprehension as described in Just and Carpenter (1992); (b) pointing out the theoretical and empirical difficulties with MacDonald and Christiansen’s alternative approach and with their simple recurrent network (SRN) model in particular; and (c)

This article deals with gradience in human sentence processing. We review the experimental evidence for the role of experience in guiding the decisions of the sentence processor. Based on this evidence, we argue that the gradient behavior observed in the processing of certain syntactic constructions can be traced back to the amount of past experience that the processor has had with these constructions. In modeling terms, linguistic experience can be approximated using large, balanced corpora. We give an overview of corpus-based and probabilistic models in the literature that have exploited this fact, and hence are well placed to make gradient predictions about processing behavior. Finally, we discuss a number of questions regarding the relationship between gradience in sentence processing and gradient grammaticality, and come to the conclusion that these two phenomena should be treated separately in conceptual and modeling terms.

This study shows experimentally how rst order recurrent neural networks learn to predict the mildly context-sensitive language MA3 = fa n b n c n ; n 1g. The training algorithm is evolutionary hill climbing with an incremental learning strategy. The experiments explore generalisation abilities of the networks emerging in the process of evolution. Generalisation with respect to depth n of the language strings and regarding their ordering in the training sequence is evaluated. Experiments without an incremental learning strategy indicate that the latter accelerates or facilitates training. Interestingly networks trained incrementally and non-incrementally show a qualitatively dierent pattern of hidden unit activity. Training on the context-free a n b 2n language and the mildly context-sensitive a n b n c n language is compared. The results show that even when the networks are identical and the data sets have the same size, learning of the context-free language is easi...

Abstract not found

Simple recurrent networks (SRNs) in symbolic time series prediction (e. g. language processing models) are frequently trained with gradient descent based learning algorithms, notably with variants of backpropagation (BP). A major drawback for the cognitive plausibility of BP is that it is a supervised scheme in which a teacher has to provide a fully specified target answer. Yet, agents in natural environments often receive a summary feedback about the degree of success or failure only, a view adopted in reinforcement learning schemes. In this work we show that for SRNs in prediction tasks for which there is a probability interpretation of the network’s output vector, Elman BP can be reimplemented as a reinforcement learning (RL) scheme for which the expected weight updates agree with the ones from traditional Elman BP. Network simulations on formal languages corroborate this result and show that the learning behaviours of Elman backpropagation (BP) and its reinforcement variant are very similar also in online learning tasks.