Results 1 
8 of
8
Recursive Distributed Representations
 Artificial Intelligence
, 1990
"... A longstanding difficulty for connectionist modeling has been how to represent variablesized recursive data structures, such as trees and lists, in fixedwidth patterns. This paper presents a connectionist architecture which automatically develops compact distributed representations for such compo ..."
Abstract

Cited by 339 (9 self)
 Add to MetaCart
A longstanding difficulty for connectionist modeling has been how to represent variablesized recursive data structures, such as trees and lists, in fixedwidth patterns. This paper presents a connectionist architecture which automatically develops compact distributed representations for such compositional structures, as well as efficient accessing mechanisms for them. Patterns which stand for the internal nodes of fixedvalence trees are devised through the recursive use of backpropagation on threelayer autoassociative encoder networks. The resulting representations are novel, in that they combine apparently immiscible aspects of features, pointers, and symbol structures. They form a bridge between the data structures necessary for highlevel cognitive tasks and the associative, pattern recognition machinery provided by neural networks. 2 J. B. Pollack 1. Introduction One of the major stumbling blocks in the application of Connectionism to higherlevel cognitive tasks, such as Na...
Learning sets of filters using back propagation
 Computer Speech and Language
, 1987
"... A learning procedure, called backpropagation, for layered networks of deterministic, neuronlike units has been described previously. The ability of the procedure automatically to discover useful internal representations makes it a powerful tool for attacking difficult problems like speech recognit ..."
Abstract

Cited by 23 (2 self)
 Add to MetaCart
A learning procedure, called backpropagation, for layered networks of deterministic, neuronlike units has been described previously. The ability of the procedure automatically to discover useful internal representations makes it a powerful tool for attacking difficult problems like speech recognition. This paper describes further research on the learning procedure and presents an example in which a network learns a set of filters that enable it to discriminate formantlike patterns in the presence of noise. The generality of the learning procedure is illustrated by a second example in which a similar network learns an edge detection task. The speed of learning is strongly dependent on the shape of the surface formed by the error measure in “weight space”. Examples are given of the error surface for a simple task and an acceleration method that speeds up descent in weight space is illustrated. The main drawback of the learning procedure is the way it scales as the size of the task and the network increases. Some preliminary results on scaling are reported and it is shown how the magnitude of the optimal weight changes depends on the fanin of the units. Additional results show how the amount of interaction between the weights affects the learning speed. The paper is concluded with a discussion of the difficulties that are likely to be encounted in applying backpropagation to more realistic problems in speech recognition. and some promising approaches to overcoming these difficulties. 1.
Group Updates and Multiscaling: An Efficient Neural Network Approach to Combinatorial Optimization
 IEEE Transactions on Systems, Man, and Cybernetics  Part B: Cybernetics
, 1996
"... A multiscale method is described in the context of binary Hopfieldtype neural networks. The appropriateness of the proposed technique for solving several classes of optimization problems is established by means of the notion of group update which is introduced here and investigated in relation to ..."
Abstract

Cited by 4 (4 self)
 Add to MetaCart
A multiscale method is described in the context of binary Hopfieldtype neural networks. The appropriateness of the proposed technique for solving several classes of optimization problems is established by means of the notion of group update which is introduced here and investigated in relation to the properties of multiscaling. The method has been tested in the solution of partitioning and covering problems, for which an original mapping to Hopfieldtype neural networks has been developed. Experimental results indicate that the multiscale approach is very effective in exploring the statespace of the problem and providing feasible solutions of acceptable quality, while at the same it offers a significant acceleration. 1 Introduction The Hopfield neural network model [7, 8] and closely related models such as the Boltzmann Machine [3, 1] have proved effective in dealing with hard optimization problems and yield nearoptimal solutions with polynomial time complexity [6, 20]. The basic ...
Adaptive TAP Equations
, 2000
"... Introduction Mean field (MF) methods provide efficient approximations which are able to cope with the increasing complexity of modern probabilistic data models. They replace the intractable task of computing high dimensional sums and integrals by the tractable problem of solving a system of nonline ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Introduction Mean field (MF) methods provide efficient approximations which are able to cope with the increasing complexity of modern probabilistic data models. They replace the intractable task of computing high dimensional sums and integrals by the tractable problem of solving a system of nonlinear equations. The TAP (21) MF approach represents a principled way for correcting the deficiencies of simple MF methods which are based on the crude approximation of replacing the intractable distribution by a factorized one, thereby neglecting important correlations between variables. In contrast, the TAP method takes into account nontrivial dependencies by estimating the reaction of all other random variables when a single variable is deleted from the system (8). The method has its origin in the statistical physics of amorphous systems, where it was developed by Thouless, Anderson and Palmer (TAP) to treat the SherringtonKirkpatrick (SK) model of disordered magnetic materials (19)
Cooperative Optimization for Energy Minimization: A Case Study of Stereo Matching
, 2007
"... Often times, individuals working together as a team can solve hard problems beyond the capability of any individual in the team. Cooperative optimization is a newly proposed general method for attacking hard optimization problems inspired by cooperation principles in team playing. It has an establis ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Often times, individuals working together as a team can solve hard problems beyond the capability of any individual in the team. Cooperative optimization is a newly proposed general method for attacking hard optimization problems inspired by cooperation principles in team playing. It has an established theoretical foundation and has demonstrated outstanding performances in solving realworld optimization problems. With some general settings, a cooperative optimization algorithm has a unique equilibrium and converges to it with an exponential rate regardless initial conditions and insensitive to perturbations. It also possesses a number of global optimality conditions for identifying global optima so that it can terminate its search process efficiently. This paper offers a general description of cooperative optimization, addresses a number of design issues, and presents a case study to demonstrate its power. I.
1 Why do we sleep?
, 2000
"... c Slowwave sleep consists in slowly recurring waves that are associated with a largescale spatiotemporal synchrony across neocortex. These slowwave complexes alternate with brief episodes of fast oscillations, similar to the sustained fast oscillations that occur during the wake state. We propos ..."
Abstract
 Add to MetaCart
c Slowwave sleep consists in slowly recurring waves that are associated with a largescale spatiotemporal synchrony across neocortex. These slowwave complexes alternate with brief episodes of fast oscillations, similar to the sustained fast oscillations that occur during the wake state. We propose that alternating fast and slow waves consolidate information acquired previously during wakefulness. Slowwave sleep would thus begin with spindle oscillations that open molecular gates to plasticity, then proceed by iteratively ‘recalling ’ and ‘storing ’ information primed in neural assemblies. This scenario provides a biophysical mechanism consistent with the growing evidence that sleep serves to consolidate memories. © 2000 Elsevier Science B.V. All rights reserved.
Schrödinger Equation As a General Optimization Algorithm
, 902
"... Abstract. One of the greatest scientific achievements of physics in the 20th century is the discovery of quantum mechanics. The Schrödinger equation is the most fundamental equation in quantum mechanics describing the timebased evolution of the quantum state of a physical system. It has been found ..."
Abstract
 Add to MetaCart
Abstract. One of the greatest scientific achievements of physics in the 20th century is the discovery of quantum mechanics. The Schrödinger equation is the most fundamental equation in quantum mechanics describing the timebased evolution of the quantum state of a physical system. It has been found that the timeindependent version of the equation can be derived from a general optimization algorithm. Instead of arguing for a new interpretation and possible deeper principle for quantum mechanics, this paper elaborates a few points of the equation as a general global optimization algorithm. Benchmarked against randomly generated hard optimization problems, this paper shows that the algorithm significantly outperformed a classic local optimization algorithm. The former found a solution in one second with a single trial better than the best one found by the latter around one hour after one hundred thousand trials. 1
A New Kind of Hopfield Networks for Finding Global Optimum
, 2005
"... Abstract — The Hopfield network has been applied to solve optimization problems over decades. However, it still has many limitations in accomplishing this task. Most of them are inherited from the optimization algorithms it implements. The computation of a Hopfield network, defined by a set of diffe ..."
Abstract
 Add to MetaCart
Abstract — The Hopfield network has been applied to solve optimization problems over decades. However, it still has many limitations in accomplishing this task. Most of them are inherited from the optimization algorithms it implements. The computation of a Hopfield network, defined by a set of difference equations, can easily be trapped into one local optimum or another, sensitive to initial conditions, perturbations, and neuron update orders. It doesn’t know how long it will take to converge, as well as if the final solution is a global optimum, or not. In this paper, we present a Hopfield network with a new set of difference equations to fix those problems. The difference equations directly implement a new powerful optimization algorithm. 1 I.