Naturally occurring speech contains only a limited amount of complex recursive structure, and this is reflected in the empirically documented difficulties that people experience when processing such structures. We present a connectionist model of human performance in processing recursive language structures. The model is trained on simple artificial languages. We find that the qualitative performance profile of the model matches human behavior, both on the relative difficulty of center-embedded and cross-dependency, and between the processing of these complex recursive structures and right-branching recursive constructions. We analyze how these differences in performance are reflected in the internal representations of the model by performing discriminant analyses on these representation both before and after training. Furthermore, we show how a network trained to process recursive structures can also generate such structures in a probabilistic fashion. This work suggests a novel expla...
|
2044
|
Learning internal representations by error propagation
– Rumelhart, G, et al.
- 1986
|
|
1140
|
Finding structure in time
– Elman
- 1990
|
|
806
|
Aspects of the Theory of Syntax
– Chomsky
- 1965
|
|
533
|
The architecture of cognition
– Anderson
- 1983
|
|
449
|
Statistical Language Learning
– Charniak
- 1997
|
|
341
|
J.M.: Signal detection theory and psychophysics
– Green, Swets
- 1966
|
|
296
|
Recursive distributed representations
– Pollack
- 1990
|
|
245
|
Theory of Syntactic Recognition for Natural Languages
– Marcus
- 1980
|
|
241
|
Learning and Development in Neural Networks: The Importance of Starting Small
– Elman
- 1993
|
|
217
|
ªDistributed Representations, Simple Recurrent Networks
– Elman
- 1991
|
|
216
|
Syntactic Structures. The Hague: Mouton
– Chomsky
- 1957
|
|
182
|
A capacity theory of comprehension: Individual differences in working memory
– Just, Carpenter
- 1992
|
|
156
|
ªLearning and Extracting Finite State Automata with SecondOrder Recurrent Neural Networks,º
– Giles, Miller, et al.
- 1992
|
|
148
|
Seven principles of surface structure parsing in natural language
– Kimball
- 1973
|
|
125
|
ªFinite State Automata and Simple Recurrent Networks,º
– Cleeremans, Servan-Schreiber, et al.
- 1989
|
|
121
|
Evidence against the context-freeness of natural language
– Shieber
- 1985
|
|
105
|
Syntactic transformations on distributed representations
– Chalmers
- 1990
|
|
102
|
Individual Choice Behavior
– Luce
- 1998
|
|
98
|
Basic methods of probabilistic context free grammars
– Jelinek, Lafferty, et al.
- 1992
|
|
72
|
Graded state machines: The representation of temporal contingencies in simple recurrent networks
– Servan-Schreiber, Cleeremans, et al.
- 1991
|
|
68
|
Individual differences in syntactic processing: the role of working memory
– King, Just
- 1991
|
|
57
|
Recursive auto-associative memory: devising compositional distributed representations
– Pollack
- 1988
|
|
50
|
Deducing linguistic structure from the statistics of large corpora
– Brill, Magerman, et al.
- 1990
|
|
47
|
Learning to segment speech using multiple cues: a connectionist model. Language and Cognitive Processes
– Christiansen, Allen, et al.
|
|
39
|
Poor estimates of context are worse than none
– Gale, Church
- 1990
|
|
34
|
Systematicity in connectionist language learning
– Hadley
- 1994
|
|
33
|
Learning to count without a counter: a case study of dynamics and activation landscapes in recurrent neural networks
– Wiles
- 1995
|
|
32
|
The finite connectivity of linguistic structure
– Stabler
- 1994
|
|
31
|
Extraction, insertion and refinement of symbolic rules in dynamically driven recurrent neural networks
– Giles, Omlin
- 1993
|
|
31
|
The Empirical Base of Linguistics: Grammaticality Judgments and Linguistic Methodology
– Schütze
- 1996
|
|
30
|
The Grammatical Basis of Linguistic Performance: Language Use and Acquisition. Current Studies in Linguistics
– Berwick, Weinberg
- 1984
|
|
29
|
Free recall of self-embedded English sentences
– Miller, Isard
- 1964
|
|
28
|
Parsnip: A connectionist network that learns natural language grammar from exposure to natural language sentences
– Hanson
- 1987
|
|
28
|
Toward connectionist parsing
– Small, Cottrell
- 1982
|
|
27
|
On memory limitations in natural language processing
– Church
- 1980
|
|
26
|
Context-free parsing in connectionist networks
– Fanty
- 1994
|
|
25
|
A language learning model for finite parameter spaces
– Niyogi, Berwick
- 1996
|
|
24
|
Infinite languages, finite minds: Connectionism, learning, and linguistic structure. Unpublished doctoral dissertation, University of
– Christiansen
- 1994
|
|
24
|
Processing crossed and nested dependencies: an automaton perspective on the psycholinguistic results
– Joshi
- 1990
|
|
22
|
Computationally Relevant Properties of Natural Languages and their Grammars
– Gazdar, Pullum
- 1985
|
|
20
|
Crossed and nested dependencies in german and dutch: A psycholinguistic study
– Bach, Brown, et al.
- 1986
|
|
19
|
Connectionist music composition based on melodic, stylistic, and psychophysical constraints
– Mozer
- 1991
|
|
18
|
A.: Some psychological studies of grammar
– Miller
- 1962
|
|
17
|
Natural Languages and Context-Free Languages
– Pullum, Gazdar
- 1982
|
|
16
|
Short-term memory limitations on decoding self-embedded sentences
– Blaubergs, Braine
- 1974
|
|
16
|
Computer science as empirical inquiry
– Newell, Simon
- 1976
|
|
16
|
On being systematically connectionist
– Niklasson, Gelder
- 1994
|
|
16
|
Grammars, parsers, and memory limitations
– Pulman
- 1986
|
|
16
|
A recurrent network that performs a contextsensitive prediction task
– Steijvers
- 1996
|
|
15
|
Memory limitations and structural forgetting: The perception of complex ungrammatical sentences as grammatical
– Gibson, Thomas
- 1999
|
