Naturally occurring speech contains only a limited amount of complex recursive structure, and this is reflected in the empirically documented difficulties that people experience when processing such structures. We present a connectionist model of human performance in processing recursive language structures. The model is trained on simple artificial languages. We find that the qualitative performance profile of the model matches human behavior, both on the relative difficulty of center-embedded and cross-dependency, and between the processing of these complex recursive structures and right-branching recursive constructions. We analyze how these differences in performance are reflected in the internal representations of the model by performing discriminant analyses on these representation both before and after training. Furthermore, we show how a network trained to process recursive structures can also generate such structures in a probabilistic fashion. This work suggests a novel expla...

this paper I put forward a new approach to understanding the origins of some of the key ingredients in a syntactic system. I show, using a computational model, that compositional syntax is an inevitable outcome of the dynamics of observationally learned communication systems. In a simulated population of individuals, language develops from a simple idiosyncratic vocabulary with little expressive power, to a compositional system with high expressivity, nouns and verbs, and word order expressing meaning distinctions.

Abstract not found

this paper. The language learning device clearly does impose constraints directly in a similar fashion --- there are certain types of language that the learner simply cannot acquire --- however these constraints are far less severe than those imposed by the LAD. As can be seen in the initial stages of the simulation, very un-language like systems can be acquired by this learner. The constraints on variation are not built into the learner, but are instead emergent properties of the social dynamics of learned communication systems and the structure of the semantic space that the individuals wish to express. The theory presented here gives us a neat explanation of why human languages use syntactic structure to compositionally derive semantics, have recursive subordination to express infinite distinctions in a digital way, have words with major syntactic categories such as noun and verb, and use structural relations (such as word order) to encode meaning distinctions. However, it does not seem to allow us to understand more specific universals. For example, why particular constituent orders are far more frequent than others across the languages of the world (Hawkins 1983; Dryer 1992). Perhaps the best explanation for these types of universal should look at the effect of parsing and generation in the transmission of replicators (see Kirby 1998a; Kirby 1997 for details). On the other hand, at least some of these word order constraints may eventually be explained in terms of linguistic adaptation without appealing to processing (see, Christiansen 1994; Christiansen & Devlin 1997 for some suggestions along these lines). X-bar theory --- a sub part of UG which constrains the structure of syntactic trees cross categorially (Jackendoff 1977) --- has been implicated in various word...

The most predominant language processing theories have, for some time, been based largely on structured knowledge and relatively simple rules. These symbolic models intentionally segregate syntactic information processing from statistical information as well as semantic, pragmatic, and discourse influences, thereby minimizing the importance of these potential constraints in learning and processing language. While such models have the advantage of being relatively simple and explicit, they are inadequate to account for learning and validated ambiguity resolution phenomena. In recent years, interactive constraint-based theories of sentence processing have gained increasing support, as a growing body of empirical evidence demonstrates early influences of various factors on comprehension performance. Connectionist networks are one form of model that naturally reflect many properties of constraint-based theories, and thus provide a form in which those theories may be instantiated. Unfortunately, most of the connectionist language models implemented until now have involved severe limitations, restricting the phenomena they could address. Comprehension and production models have, by and large, been limited to simple sentences with small vocabularies (cf. St. John & McClelland, 1990). Most models that have addressed the problem of complex, multi-clausal sentence processing have been prediction networks (cf. Elman, 1991; Christiansen & Chater, 1999a). Although a useful component of a language processing system, prediction does not get at the heart of language: the interface between syntax and semantics.

This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and recent developments in speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models are likely to have an important influence on future research. Introduction Connectionist modeling of language processing has been highly controversial. Some have argued that language proce...

John Maynard Smith and Eörs Szathmáry argued that human language signified the eighth major transition in evolution: human language marked a new form of information transmission from one generation to another [Maynard Smith J, Szathmáry E. The major transitions in evolution. Oxford: Oxford Univ. Press; 1995]. According to this view language codes cultural information and as such forms the basis for the evolution of complexity in human culture. In this article we develop the theory that language also codes information in another sense: languages code information on their own structure. As a result, languages themselves provide information that influences their own survival. To understand the consequences of this theory we discuss recent computational models of linguistic evolution. Linguistic evolution is the process by which languages themselves evolve. This article draws together this recent work on linguistic evolution and highlights the significance of this process in understanding the evolution of linguistic complexity. Our conclusions are that: (1) the process of linguistic transmission constitutes the basis for an evolutionary system, and (2), that this evolutionary system is only superficially comparable to the process of

Abstract Many artificial life researchers stress the interdisciplinary character of the field. Against such a backdrop, this report reviews and discusses artificial life, as it is depicted in, and as it interfaces with, adjacent disciplines (in particular, philosophy, biology, and linguistics), and in the light of a specific historical example of interdisciplinary research (namely cybernetics) with which artificial life shares many features. This report grew out of a workshop held at the Sixth European Conference on Artificial Life in Prague and features individual contributions from the workshop’s eight speakers, plus a section designed to reflect the debates that took place during the workshop’s discussion sessions. The major theme that emerged during these sessions was the

It is widely assumed that human learning and the structure of human languages are intimately related. This relationship is frequently suggested to be rooted in a language-specific biological endowment, which encodes universal, but arbitrary, principles of language structure (a universal grammar or UG). How might such a UG have evolved? We argue that UG could not have arisen either by biological adaptation or non-adaptationist genetic processes. The resulting puzzle concerning the origin of UG we call the logical problem of language evolution. Because the processes of language change are much more rapid than processes of genetic change, language constitutes a “moving target ” both over time and across different human populations, and hence cannot provide a stable environment to which UG genes could have adapted. We conclude that a biologically determined UG is not evolutionarily viable. Instead, the original motivation for UG—the mesh between learners and languages—arises because language has been shaped to fit the human brain, rather than vice versa. Following Darwin, we view language itself as a complex and interdependent “organism, ” which evolves under selectional pressures from human learning and processing mechanisms. That is, languages are themselves undergoing severe selectional pressure from each generation of language users and learners. This suggests that apparently arbitrary aspects of linguistic structure may result from general learning and processing biases, independent of language. We illustrate how this framework can integrate evidence from different literatures and methodologies to explain core linguistic phenomena, including binding constraints, word order universals, and diachronic language change. 1.

Complex words consist of morphemic subunits that can recombine to form other words. Thus midnight is standardly analyzed as consisting of the prefix mid- and stem night, which also occur in words such as midstream and nightly. A considerable body of empirical and theoretical research suggests that morphological structure governs the representation of words in memory and that many words are decomposed into morphological components in processing. We investigated an alternative approach in which morphology arises from the interaction of semantic, phonological, and orthographic codes. Five cross-modal lexical decision experiments show that the magnitude of priming (e.g., for pairs such as teacher-teach) is affected by the degree of semantic and phonological overlap between words. Crucially, items that are only moderately similar produce intermediate facilitation effects (e.g., latelylate) . This pattern is observed both for words standardly treated as morphologically related (e.g., teacher-teach) and for morphologically unrelated words that exhibit similar degrees of semantic and phonological overlap (e.g., snarl-sneer). The results can be understood in terms of connectionist models employing distributed representations rather than discrete morphemes. Graded semantic and phonological similarity effects in priming: Evidence for a distributed connectionist approach to morphology One of the fundamental problems in the study of language is to characterize knowledge of words and how this knowledge is used in comprehension and production. The focus of the present article is on derivational morphology, the aspect of lexical knowledge concerning the structure and formation of complex words. Words such as baker and talking appear to consist of components, traditionally called m...