An approach to connectionist natural language processing is proposed, which is based on hierarchically organized modular Parallel Distributed Processing (PDP) networks and a central lexicon of distributed input/output representations. The modules communicate using these representations, which are global and publicly available in the system. The representations are developed automatically by all networks while they are learning their processing tasks. The resulting representations reflect the regularities in the subtasks, which facilitates robust processing in the face of noise and damage, supports improved generalization, and provides expectations about possible contexts. The lexicon can be extended by cloning new instances of the items, that is, by generating a number of items with known processing properties and distinct identities. This technique combinatorially increases the processing power of the system. The recurrent FGREP module, together with a central lexicon, is used as a ba...

A number of connectionist models capable of representing data with compositional structure have recently appeared. These new models suggest the intriguing possibility of performing holistic structure-sensitive computations with distributed representations. Two possible forms of holistic inference, transformational inference and confluent inference, are identified and compared. Transformational inference was successfully demonstrated in [Chalmers, 1990]; however, since the pure transformational approach does not consider the eventual inference tasks during the process of learning its representations, there is a drawback that the holistic transformation corresponding to a given inference task could become arbitrarily complex, and thus very difficult to learn. Confluent inference addresses this drawback by achieving a tight coupling between the distributed representations of a problem and the solution for the given inference task while the net is still learning its representations. A dual...

This paper describes a connectionist system for representing and reasoning with multiple inheritance structures with exceptions. The representational architecture has three characteristics. First, it merges relational with taxonomic representations. Secondly, it handles conflicts generated by exceptions and the use of multiple superclasses. Thirdly, it uses fully distributed representations. One novel feature is that, since the distributed representation of an entity is influenced by its position in the inheritance structure, representations of assertions are influenced by the context of the entities. An extension to the model which implements and makes use of confluent inference is described. 1 The problem of structured representation Ideally, object-centred representations (e.g. frames, semantic networks) should be hierarchic, with subclasses and instances having no more than one class to which they belong. But commonsense reasoning is littered with examples of multiple inheritance,...

Pure exemplar-based learning suffers from the inability to choose an appropriate generalisation out of many possible, where each is correct with respect to the learning examples. In connectionist terms; learning is not necessarily successful for input data not included in the learning examples even if the learning error is minimized. Ideas from research in machine learning, artificial intelligence and connectionism are used to develop intuitions for a biased approach where knowledge about the type, its future use and simplicity of the implementation may play a part in deciding which generalisation to prefer. 1 Motivation Many connectionists have concentrated on learning as if it was a matter of detecting and discovering statistical variances and regularities. In this paper a different standpoint is taken. It is argued that the detection of regularities is not sufficient to guarantee success. Parametric and data based methods are inherently not intelligent. It is argued that we need kn...

The overall aim of the paper is to demonstrate that, from a machine learning point of view, connectionist networks are not black boxes. Trained networks contain rich and varied internal representations gleaned from training sets. Analysis of these representations can provide useful results concerning the generalizability of these networks to novel examples. More particularly in the case of common sense reasoning, hyperplane analysis can demonstrate the adaptive power of connectionist networks when presented with information concerning new entities and the ability of such networks to cope with exceptions to what has already been input in the training set. One important advantage of such analysis is that machine learning researchers need not be committed to any particular form of symbolic structure but can study the process of acquiring common sense reasoning as well as the impact of new information on existing internal representations purely from the viewpoint of learning. Another advan...

The overall aim of the paper is to demonstrate that, from a machine learning point of view, connectionist networks are not black boxes. Trained networks contain rich and varied internal representations gleaned from training sets. Analysis of these representations can provide useful results concerning the generalizability of these networks to novel examples. More particularly in the case of common sense reasoning, the recently developed technique of hyperplane analysis 1 can demonstrate the adaptive power of connectionist networks when presented with information concerning new entities and the ability of such networks to cope with exceptions to what has already been input in the training set. One important advantage of such analysis is that machine learning researchers need not be committed to any particular form of symbolic structure but can study the process of acquiring common sense reasoning as well as the impact of new information on existing internal representations purely from ...