Abstract. In this paper three problems for a connectionist account of language are considered: 1. What is the nature of linguistic representations? 2. How can complex structural relationships such as constituent structure be represented? 3. How can the apparently open-ended nature of language be accommodated by a fixed-resource system? Using a prediction task, a simple recurrent network (SRN) is trained on multiclausal sentences which contain multiply-embedded relative clauses. Principal component analysis of the hidden unit activation patterns reveals that the network solves the task by developing complex distributed representations which encode the relevant grammatical relations and hierarchical constituent structure. Differences between the SRN state representations and the more traditional pushdown store are discussed in the final section.

An approach to connectionist natural language processing is proposed, which is based on hierarchically organized modular Parallel Distributed Processing (PDP) networks and a central lexicon of distributed input/output representations. The modules communicate using these representations, which are global and publicly available in the system. The representations are developed automatically by all networks while they are learning their processing tasks. The resulting representations reflect the regularities in the subtasks, which facilitates robust processing in the face of noise and damage, supports improved generalization, and provides expectations about possible contexts. The lexicon can be extended by cloning new instances of the items, that is, by generating a number of items with known processing properties and distinct identities. This technique combinatorially increases the processing power of the system. The recurrent FGREP module, together with a central lexicon, is used as a ba...

This paper deals with the problem of variable binding in connectionist networks. Specifically, a more thorough solution to the variable binding problem based on the Discrete Neuron formalism is proposed and a number of issues arising in the solution are examined in relation to logic: consistency checking, binding generation, unification, and functions. We analyze what is needed in order to resolve these issues, and based on this analysis, a procedure is developed for systematically setting up connectionist networks for variable binding based on logic rules. This solution compares favorably to similar solutions in simplicity and completeness. ACKNOWLEDGEMENTS. I wish to thank Dave Waltz, James Pustejovsky, and Tim Hickey for many discussions that helped me to elucidate ideas contained in this paper. I am also grateful to the three anonymous reviewers for their insightful criticisms and useful suggestions. 2 1 Introduction When discussing connectionist models in relation to reasoning...

This paper, however, is concerned more with general approaches, ideas and possibilities than with precise technical details (which can be found elsewhere, as will be pointed out along the way)

This paper presents a survey of connectionist inference systems.

Connectionism the use of neural networks for knowledge representation and inference has profound implications for the representation and processing of information because it provides a fundamentally new view of knowledge. However, its progress is impeded by the lack of a unifying theoretical construct corresponding to the idea of a calculus (or formal system) in traditional ap- proaches to knowledge representation. Such a construct, called a simulacrum, is proposed here, and its basic properties are explored. We find that although exact classification is impossible, several other useful, robust kinds of classification are permitted. The representation of structured information and constituent structure are considered, and we find a basis for more flexible rule-like processing than that permitted by conventional methods. We discuss briefly logical issues such as decidability and computability and show that they require reformulation in this new context. Throughout we discuss the implications for artificial intelligence and cognitive science of this new theoretical framework.

We investigate connectionist models of rule-based reasoning, and show that while such models usually carry out reasoning in exactly the same way as symbolic systems, they have more to offer in terms of commonsense reasoning. A connectionist architecture for commonsense reasoning, CONSYDERR, is proposed to account for commonsense reasoning patterns and to remedy the brittleness problem in traditional rule-based systems. A dual representational scheme is devised, utilizing both localist and distributed representations and exploring the synergy resulting from the interaction between the two. CONSYDERR is therefore capable of accounting for many difficult patterns in commonsense reasoning. This work shows that connectionist models of reasoning are not just "implementations" of their symbolic counterparts, but better computational models of commonsense reasoning. Introduction Rule-based reasoning is the most prominent paradigm of symbolic AI. Whether connectionist models...

We present a scheme for translating high-level descriptions of conceptual hierarchies into a neural network representation. The intuitive semantics of a conceptual hierarchy is provided by a Bayesian net, and the neural network implementation provably approximates the behaviour of this net under a stochastic simulation rule.

This paper demonstrates the role of connectionist (neural network) models in reasoning beyond that of an associative memory. First we show that there is a connection between propositional logics and the weighted-sum computation customarily used in connectionist models. Specifically, the weighted-sum computation can handle Horn clause logic and Shoham's logic as special cases. Secondly, we show how variables can be incorporated into connectionist models to enhance their representational power. We devise solutions to the connectionist variable binding problem to enable connectioninst networks to handle variables and dynamic bindings in reasoning. A new model, the Discrete Neuron formalism, is employed for dealing with the variable binding problem, which is an extension of the weighted-sum models. Formal definitions are presented, and examples are analyzed in details. ACKNOWLEDGEMENTS I wish to thank Dave Waltz, James Pustejovsky, and Tim Hickey (all of Brandeis University) for many discu...

The paper attempts to explore high-level connectionist models for approximate commonsense reasoning from a broad perspective, investigating their connections to two other prominent paradigms: role-based reasoning and case-based reasoning. High-level connectionnist models, especially the CONSYDERR architecture, are studied in light of these paradigms.