Results 1  10
of
12
Recursive Distributed Representations
 Artificial Intelligence
, 1990
"... A longstanding difficulty for connectionist modeling has been how to represent variablesized recursive data structures, such as trees and lists, in fixedwidth patterns. This paper presents a connectionist architecture which automatically develops compact distributed representations for such compo ..."
Abstract

Cited by 339 (9 self)
 Add to MetaCart
A longstanding difficulty for connectionist modeling has been how to represent variablesized recursive data structures, such as trees and lists, in fixedwidth patterns. This paper presents a connectionist architecture which automatically develops compact distributed representations for such compositional structures, as well as efficient accessing mechanisms for them. Patterns which stand for the internal nodes of fixedvalence trees are devised through the recursive use of backpropagation on threelayer autoassociative encoder networks. The resulting representations are novel, in that they combine apparently immiscible aspects of features, pointers, and symbol structures. They form a bridge between the data structures necessary for highlevel cognitive tasks and the associative, pattern recognition machinery provided by neural networks. 2 J. B. Pollack 1. Introduction One of the major stumbling blocks in the application of Connectionism to higherlevel cognitive tasks, such as Na...
On the Complexity of Computing and Learning with Multiplicative Neural Networks
 NEURAL COMPUTATION
"... In a great variety of neuron models neural inputs are combined using the summing operation. We introduce the concept of multiplicative neural networks that contain units which multiply their inputs instead of summing them and, thus, allow inputs to interact nonlinearly. The class of multiplicative n ..."
Abstract

Cited by 24 (3 self)
 Add to MetaCart
In a great variety of neuron models neural inputs are combined using the summing operation. We introduce the concept of multiplicative neural networks that contain units which multiply their inputs instead of summing them and, thus, allow inputs to interact nonlinearly. The class of multiplicative neural networks comprises such widely known and well studied network types as higherorder networks and product unit networks. We investigate the complexity of computing and learning for multiplicative neural networks. In particular, we derive upper and lower bounds on the VapnikChervonenkis (VC) dimension and the pseudo dimension for various types of networks with multiplicative units. As the most general case, we consider feedforward networks consisting of product and sigmoidal units, showing that their pseudo dimension is bounded from above by a polynomial with the same order of magnitude as the currently best known bound for purely sigmoidal networks. Moreover, we show that this bound holds even in the case when the unit type, product or sigmoidal, may be learned. Crucial for these results are calculations of solution set components bounds for new network classes. As to lower bounds we construct product unit networks of fixed depth with superlinear VC dimension. For sigmoidal networks of higher order we establish polynomial bounds that, in contrast to previous results, do not involve any restriction of the network order. We further consider various classes of higherorder units, also known as sigmapi units, that are characterized by connectivity constraints. In terms of these we derive some asymptotically tight bounds.
Constraint, Word Frequency, and the Relationship between Lexical Processing Levels in Spoken Word Production
, 1998
"... this report was submitted about how the processes are affected by fre to fulfill the requirements for a Masters degree from the quency and constraint ..."
Abstract

Cited by 19 (2 self)
 Add to MetaCart
this report was submitted about how the processes are affected by fre to fulfill the requirements for a Masters degree from the quency and constraint
Learning by Gradient Descent in Function Space
 In Proc. of IEEE Int'l Conf. on System, Man, and Cybernetics
, 1990
"... : Traditional connectionist networks have homogeneous nodes wherein each node executes the same function. Networks where each node executes a different function can be used to achieve efficient supervised learning. A modified backpropagation algorithm for such networks, which performs gradient desc ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
: Traditional connectionist networks have homogeneous nodes wherein each node executes the same function. Networks where each node executes a different function can be used to achieve efficient supervised learning. A modified backpropagation algorithm for such networks, which performs gradient descent in "function space," is presented and its advantages are discussed. The benefits of the suggested paradigm include faster learning and ease of interpretation of the trained network. 1 Introduction Connectionist networks (Rosenblatt, 1962; Grossberg, 1981; Rumelhart, McClelland & the PDP Research Group, 1986) are usually thought of to be graphlike interconnections of processing elements. Each processing element is capable of a simple computation such as summation or summation coupled with thresholding. In such networks, the delta rule has been used for learning linear concepts and the generalized delta rule (Rumelhart, McClelland & the PDP Research Group, 1986) has been employed for lea...
Encoding A Priori Information In Feedforward Networks
, 1991
"... Theoretical results and practical experience indicate that feedforward networks are very good at approximating a wide class of functional relationships. Training networks to approximate functions takes place by using exemplars to find interconnect weights that maximize some goodness of fit criterion ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
Theoretical results and practical experience indicate that feedforward networks are very good at approximating a wide class of functional relationships. Training networks to approximate functions takes place by using exemplars to find interconnect weights that maximize some goodness of fit criterion. Given finite data sets it can be important in the training process to take advantage of any a priori information regarding the underlying functional relationship to improve the approximation and the ability of the network to generalize. This paper describes methods for incorporating a priori information of this type into feedforward networks. Two general approaches, one based upon architectural constraints and a second upon connection weight constraints form the basis of the methods presented. These two approaches can be used either alone or in combination to help solve specific training problems. Several examples covering a variety of types of a priori information, including information a...
Connectionist Cognitive Processing for Invariant Pattern Recognition
"... Recently, from classical connectionist and symbolic models, the new field of neurosymbolic integration has emerged, whose aim is to benefit from the advantages of both domains to model human perceptive and cognitive capabilities. To reach this goal, some strategies are envisaged, among which connect ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Recently, from classical connectionist and symbolic models, the new field of neurosymbolic integration has emerged, whose aim is to benefit from the advantages of both domains to model human perceptive and cognitive capabilities. To reach this goal, some strategies are envisaged, among which connectionist cognitive processing claims that these desired capabilities can emerge from pure neuronal structures and processes. This approach refers to the substratum of cognition, the human brain and gives rise to perceptually grounded models whose goal is to reach higher cognitive levels. Its principles are presented here and, as an illustration, an application to invariant pattern recognition is described. From basic connectionist models, a biologically inspired model of neuronal networks cooperation is implemented to allow for internal information translation. This mechanism leads to automatic pattern centring in a classical character recognition application with excellent performances. Intr...
Boolean Functions and Artificial Neural Networks
 Department of Mathematics and Centre for Discrete and Applicable Mathematics, The London School of Economics and Political Science
, 2003
"... This report surveys some connections between Boolean functions and artificial neural networks. The focus is on cases in which the individual neurons are linear threshold neurons, sigmoid neurons, polynomial threshold neurons, or spiking neurons. We explore the relationships between types of artif ..."
Abstract
 Add to MetaCart
This report surveys some connections between Boolean functions and artificial neural networks. The focus is on cases in which the individual neurons are linear threshold neurons, sigmoid neurons, polynomial threshold neurons, or spiking neurons. We explore the relationships between types of artificial neural network and classes of Boolean function. In particular, we investigate the type of Boolean functions a given type of network can compute, and how extensive or expressive the set of functions so computable is. A version of this is to appear as a chapter in a book on Boolean functions, but the report itself is relatively selfcontained.
Connections between Neural Networks and Boolean Functions ∗
"... This report surveys some connections between Boolean functions and artificial neural networks. The focus is on cases in which the individual neurons are linear threshold neurons, sigmoid neurons, polynomial threshold neurons, or spiking neurons. We explore the relationships between types of artifici ..."
Abstract
 Add to MetaCart
This report surveys some connections between Boolean functions and artificial neural networks. The focus is on cases in which the individual neurons are linear threshold neurons, sigmoid neurons, polynomial threshold neurons, or spiking neurons. We explore the relationships between types of artificial neural network and classes of Boolean function. In particular, we investigate the type of Boolean functions a given type of network can compute, and how extensive or expressive the set of functions so computable is. To appear as a chapter in Boolean Methods and Models (ed. Yves Crama and Peter L. Hammer).