Abstract. In this paper three problems for a connectionist account of language are considered: 1. What is the nature of linguistic representations? 2. How can complex structural relationships such as constituent structure be represented? 3. How can the apparently open-ended nature of language be accommodated by a fixed-resource system? Using a prediction task, a simple recurrent network (SRN) is trained on multiclausal sentences which contain multiply-embedded relative clauses. Principal component analysis of the hidden unit activation patterns reveals that the network solves the task by developing complex distributed representations which encode the relevant grammatical relations and hierarchical constituent structure. Differences between the SRN state representations and the more traditional pushdown store are discussed in the final section.

Does knowledge of language consist of mentally-represented rules? Rumelhart and McClelland have described a connectionist (parallel distributed processing) model of the acquisition of the past tense in English which successfully maps many stems onto their past tense forms, both regular (walk/walked) and irregular (go/went), and which mimics some of the errors and sequences of development of children. Yet the model contains no explicit rules, only a set of neuron-style units which stand for trigrams of phonetic features of the stem, a set of units which stand for trigrams of phonetic features of the past form, and an array of connections between the two sets of units whose strengths are modified during learning. Rumelhart and McClelland conclude that linguistic rules may be merely convenient approximate fictions and that the real causal processes in language use and acquisition must be characterized as the transfer of activation levels among units and the modification of the weights of their connections. We analyze both the linguistic and the developmental assumptions of the model in detail and discover that (1) it cannot represent certain words, (2) it cannot learn many rules, (3) it can learn rules found in no human language, (4) it cannot explain morphological and phonological regularities, (5) it cannot explain the differences between irregular and regular forms, (6) it fails at its assigned task of mastering the past tense of English, (7) it gives an incorrect explanation for two developmental phenomena: stages of overregularization of irregular forms such as bringed, and the appearance of doubly-marked forms such as ated, and (8) it gives accounts of two others (infrequent overregularization of verbs ending in t/d, and the order of acquisition of different irregula...

A higher order recurrent neural network architecture learns to recognize and generate languages after being "trained " on categorized exemplars. Studying these networks from the perspective of dynamical systems yields two interesting discoveries: First, a longitudinal examination of the learning process illustrates a new form of mechanical inference: Induction by phase transition. A small weight adjustment causes a "bifurcation" in the limit behavior of the network. This phase transition corresponds to the onset of the network’s capacity for generalizing to arbitrary-length strings. Second, a study of the automata resulting from the acquisition of previously published training sets indicates that while the architecture is not guaranteed to find a minimal finite automaton consistent with the given exemplars, which is an NP-Hard problem, the architecture does appear capable of generating non-regular languages by exploiting fractal and chaotic dynamics. I end the paper with a hypothesis relating linguistic generative capacity to the behavioral regimes of non-linear dynamical systems.

We present results from an experiment similar to one performed by Packard [24], in which a genetic algorithm is used to evolve cellular automata (CA) to perform a particular computational task. Packard examined the frequency of evolved CA rules as a function of Langton’s λ parameter [17], and interpreted the results of his experiment as giving evidence for the following two hypotheses: (1) CA rules able to perform complex computations are most likely to be found near “critical ” λ values, which have been claimed to correlate with a phase transition between ordered and chaotic behavioral regimes for CA; (2) When CA rules are evolved to perform a complex computation, evolution will tend to select rules with λ values close to the critical values. Our experiment produced very different results, and we suggest that the interpretation of the original results is not correct. We also review and discuss issues related to λ, dynamical-behavior classes, and computation in CA. The main constructive results of our study are identifying the emergence and competition of computational strategies and analyzing the central role of symmetries in an evolutionary system. In particular, we demonstrate how symmetry breaking can impede the evolution toward higher computational capability.

Distributed representations are attractive for a number of reasons. They offer the possibility of representing concepts in a continuous space, they degrade gracefully with noise, and they can be processed in a parallel network of simple processing elements. However, the problem of representing nested structure in distributed representations has been for some time a prominent concern of both proponents and critics of connectionism [Fodor and Pylyshyn 1988; Smolensky 1990; Hinton 1990]. The lack of connectionist representations for complex structure has held back progress in tackling higher-level cognitive tasks such as language understanding and reasoning. In this thesis I review connectionist representations and propose a method for the distributed representation of nested structure, which I call "Holographic Reduced Representations " (HRRs). HRRs provide an implementation of Hinton's [1990] "reduced descriptions". HRRs use circular convolution to associate atomic items, which are rep...

Abstract not found

Questions about how our knowledge changes in respOnse to experience lie at the heart of efforts to understand cognitive development, In this chapter, I approach these questions from a connectio.ust perspective, I contrast a connectionist approach to these questions with traditional symbolic or propositional approaches, I suggest that thinking a~ut the development of knowledge has been heavily influenced by the assumption that knowledge is symbolic, and I argue that a connectionist approach leads to new conceptualizations of the processes through which developing children come to know lIlore and more about the world" These issues are explored by considering a connectionist simulation " model that is',appliedto tbebalance scale task studiecJ by Siegler and otbers, The graded ' nature or the representationS used by, tbelllodel. allows it. to account for several aspects of the empirical data, including the Torque Difference Effect (Ferretti & Butterfield,. 1986), The incremental nature of connectionist learning-the fact that current learning builds ' on what. has already been learned-allows the model to account for stagelike developmental progressions and for differences readiness to, learn from particular experiences at different points in development, The chapter also shows bow the connectionist framework allows one to capture effects of ' cue,complexity as weD as. cue familiarity on the course of development, The discussion considers tIJe essential features of the connectionist account of performance and development in the balance scale task, and considers open questions, such as the nature of the initial constraints necessary to lead to successful development, and the relation-

The purpose of this book is to present a collection of papers that represents a broad spectrum of current research in learning methods for natural language processing, and to advance the state of the art in language learning and artificial intelligence. The book should bridge a gap between several areas that are usually discussed separately, including connectionist, statistical, and symbolic methods. In order to bring together new and different language learning approaches, we held a workshop at the International Joint Conference on Artificial Intelligence in Montreal in August 1995. Paper contributions were selected and revised after having been reviewed by at least twomembers of the international program committee as well as additional reviewers. This book contains the revised workshop papers and additional papers by members of the program committee. In particular this book focuses on current issues such as: -- How can we apply existing learning methods to language processing? -- What new learning methods are needed for language processing and why? -- What language knowledge should be learned and why?

This paper is about the philosophical and methodological foundation of artificial intelligence (AI). After discussing what is a good "working definition", "intelligence" is defined as "the ability for an information processing system to adapt to its environment with insufficient knowledge and resources". Applying the definition to a reasoning system, we get the major components of Non-Axiomatic Reasoning System (NARS), which isasymbolic logic implemented in a computer system, and has many interesting properties that are closely related to intelligence. The definition also clari es the difference and relationship between AI and other disciplines, such as computer science. Finally, the definition is compared with other popular definitions of intelligence, and its advantages are argued.

This paper assesses the current status of Piaget's theory of sensorimotor intelligence in relation to three persistent issues about the abilities of human infants: the nature of initial mechanisms � the traditional view that re-presentational functioning is the outcome of infant development � and the place of generalpurpose developmental processes. Varela's view of three successive paradigms for cognitive science | cognitivism, emergence and enaction | is introduced as a means for locating Piaget's ideas on action and epigenesis in relation to approaches of particular relevance to understanding infancy. The contribution of work that aims to understand how situated systems can be organized to function as autonomous agents exhibiting adaptive behaviour is considered through examples of computational work in behaviour-based robotics. This supports Piaget's stress on action, but challenges his assumptions about the outcome of infant development. Finally, the relevance to infancy, and to Piaget's theory, of Karmilo-Smith's proposals for cognitive development through a process of