In this paper three problems for a connectionist account of language are considered:
1. What is the nature of linguistic representations?
2. How can complex structural relationships such as constituent structure be represented?
3. How can the apparently open-ended nature of language be accommodated by a fixed-resource system?
Using a prediction task, a simple recurrent network (SRN) is trained on multiclausal sentences which contain multiply-embedded relative clauses. Principal component analysis of the hidden unit activation patterns reveals that the network solves the task by developing complex distributed representations which encode the relevant grammatical relations and hierarchical constituent structure. Differences between the SRN state representations and the more traditional pushdown store are discussed in the final section.
Using a prediction task, a simple recurrent network (SRN) is trained on multiclausal sentences which contain multiply-embedded relative clauses. Principal component analysis of the hidden unit activation patterns reveals that the network solves the task by developing complex distributed representations which encode the relevant grammatical relations and hierarchical constituent structure. Differences between the SRN state representations and the more traditional pushdown store are discussed in the final section.
|
2044
|
Learning internal representations by error propagation
– Rumelhart, G, et al.
- 1986
|
|
1140
|
Finding structure in time
– Elman
- 1990
|
|
663
|
Language identification in the limit
– Gold
- 1967
|
|
429
|
Connectionism and cognitive architecture: A critical analysis
– Fodor
- 1988
|
|
383
|
Parallel networks that learn to pronounce English text
– Sejnowski, Rosenberg
- 1987
|
|
293
|
The Language of Thought
– Fodor
- 1975
|
|
260
|
On the proper treatment of connectionism
– Smolensky
- 1988
|
|
257
|
On learning the past tense of English verbs
– Rumelhart
- 1986
|
|
231
|
dangerous things: What categories reveal about the mind. Chicago: Univ
– Lakoff
- 1987
|
|
217
|
Digital Image Processing
– Gonzalez, Woods
- 1992
|
|
216
|
Syntactic Structures. The Hague: Mouton
– Chomsky
- 1957
|
|
216
|
Learnability and Cognition: The Acquisition of Argument Structure
– Pinker
- 1989
|
|
203
|
Connectionist models and their properties
– Feldman, Ballard
- 1982
|
|
179
|
Distributed representations
– Hinton, McClelland, et al.
- 1986
|
|
154
|
ªSerial Order: A Parallel Distributed Processing Approach,º
– Jordan
- 1986
|
|
107
|
Skeletonization: A technique for trimming the fat from a network via relevance assessment
– Mozer, Smolensky
- 1989
|
|
105
|
Syntactic transformations on distributed representations
– Chalmers
- 1990
|
|
98
|
Mapping part-whole hierarchies in connectionist networks
– Hinton
- 1990
|
|
75
|
Frame semantics
– Fillmore
- 1982
|
|
72
|
Graded state machines: The representation of temporal contingencies in simple recurrent networks
– Servan-Schreiber, Cleeremans, et al.
- 1991
|
|
64
|
ªA Focused Backpropagation Algorithm for Temporal Pattern Recognition,º
– Mozer
- 1989
|
|
63
|
Mental spaces
– Fauconnier
- 1994
|
|
61
|
Reading senseless sentences: brain potentials reflect semantic incongruity
– Kutas, SA
- 1980
|
|
59
|
A spreading activation theory of retrieval in sentence production
– Dell
- 1986
|
|
53
|
The temporal structure of spoken language understanding
– Marslen-Wilson
- 1980
|
|
50
|
Representation and structure in connectionist models
– Elman
- 1991
|
|
46
|
Syntax: A functional-typological introduction, Volume 1. Benjamins
– Givon
- 1984
|
|
33
|
Functionalist approaches to grammar
– Bates
- 1982
|
|
33
|
Language learning: cues or rules
– MacWhinney, Leinbach, et al.
- 1989
|
|
32
|
Common Principal Components and Related Multivariate Models
– Flury
- 1988
|
|
30
|
Levels of processing and the structure of the language processor
– Forster
- 1979
|
|
29
|
Spoken word recognition processes and the gating paradigm. Perception and Psychophysics 28.267--283
– GROSJEAN
- 1980
|
|
29
|
Functional Syntax: Anaphora, Discourse and Empathy
– Kuno
- 1987
|
|
29
|
Sentence comprehension: A parallel distributed processing approach
– John
- 1989
|
|
28
|
Meaning and structure of language
– Chafe
- 1970
|
|
28
|
Universal approximation using feedforward networks with on-sigmoid hidden layer activation functions
– Stinchcombe, White
- 1989
|
|
28
|
Symbols among the neurons: Details of a connectionist inference architecture
– Touretzky, Hinton
- 1985
|
|
25
|
Syntactic Theory and the Projection Problem
– Baker
- 1979
|
|
25
|
Transitivity in Grammar and Discourse
– Hopper, Thompson
- 1980
|
|
24
|
On Variable Binding and the Representation of Symbolic Structures in Connectionist Systems
– Smolensky
- 1987
|
|
22
|
A modular neural network architecture for sequential paraphrasing of script-based stories
– Miikkulainen, Dyer
- 1989
|
|
22
|
Pdp models and general issues in cognitive science
– Rumelhart, McClelland
- 1986
|
|
21
|
Distributed representations of ambiguous words and their resolution in a connectionist network
– Kawamoto
- 1988
|
|
21
|
Boltzcons: Reconciling connectionism with the recursive nature of stacks and trees
– Touretzky
- 1986
|
|
17
|
The case for interactionism in language processing
– McClelland
- 1987
|
|
16
|
ªThe Role of Similarity in Hungarian Vowel Harmony: A Connectionist Account,º
– Hare
- 1990
|
|
15
|
Encoding input/output representations in connectionist cognitive systems
– Miikkulainen, Dyer
- 1989
|
|
14
|
A dynamic usage-based model
– Langacker
- 2000
|
|
14
|
On the proper treatment of connectionism. Behavioral and Brain Sciences
– Smolensky
- 1988
|
|
14
|
A study of the ability to decode grammatically novel sentences
– Stolz
- 1967
|
|
