A long-standing difficulty for connectionist modeling has been how to represent variable-sized recursive data structures, such as trees and lists, in fixed-width patterns. This paper presents a connectionist architecture which automatically develops compact distributed representations for such compositional structures, as well as efficient accessing mechanisms for them. Patterns which stand for the internal nodes of fixed-valence trees are devised through the recursive use of back-propagation on three-layer autoassociative encoder networks. The resulting representations are novel, in that they combine apparently immiscible aspects of features, pointers, and symbol structures. They form a bridge between the data structures necessary for high-level cognitive tasks and the associative, pattern recognition machinery provided by neural networks. 2 J. B. Pollack 1. Introduction One of the major stumbling blocks in the application of Connectionism to higherlevel cognitive tasks, such as Na...

A learning procedure, called back-propagation, for layered networks of deterministic, neuron-like units has been described previously. The ability of the procedure automatically to discover useful internal representations makes it a powerful tool for attacking difficult problems like speech recognition. This paper describes further research on the learning procedure and presents an example in which a network learns a set of filters that enable it to discriminate formant-like patterns in the presence of noise. The generality of the learning procedure is illustrated by a second example in which a similar network learns an edge detection task. The speed of learning is strongly dependent on the shape of the surface formed by the error measure in “weight space”. Examples are given of the error surface for a simple task and an acceleration method that speeds up descent in weight space is illustrated. The main drawback of the learning procedure is the way it scales as the size of the task and the network increases. Some preliminary results on scaling are reported and it is shown how the magnitude of the optimal weight changes depends on the fan-in of the units. Additional results show how the amount of interaction between the weights affects the learning speed. The paper is concluded with a discussion of the difficulties that are likely to be encounted in applying back-propagation to more realistic problems in speech recognition. and some promising approaches to overcoming these difficulties. 1.

A multiscale method is described in the context of binary Hopfield--type neural networks. The appropriateness of the proposed technique for solving several classes of optimization problems is established by means of the notion of group update which is introduced here and investigated in relation to the properties of multiscaling. The method has been tested in the solution of partitioning and covering problems, for which an original mapping to Hopfield-type neural networks has been developed. Experimental results indicate that the multiscale approach is very effective in exploring the state-space of the problem and providing feasible solutions of acceptable quality, while at the same it offers a significant acceleration. 1 Introduction The Hopfield neural network model [7, 8] and closely related models such as the Boltzmann Machine [3, 1] have proved effective in dealing with hard optimization problems and yield near-optimal solutions with polynomial time complexity [6, 20]. The basic ...

Introduction Mean field (MF) methods provide efficient approximations which are able to cope with the increasing complexity of modern probabilistic data models. They replace the intractable task of computing high dimensional sums and integrals by the tractable problem of solving a system of nonlinear equations. The TAP (21) MF approach represents a principled way for correcting the deficiencies of simple MF methods which are based on the crude approximation of replacing the intractable distribution by a factorized one, thereby neglecting important correlations between variables. In contrast, the TAP method takes into account nontrivial dependencies by estimating the reaction of all other random variables when a single variable is deleted from the system (8). The method has its origin in the statistical physics of amorphous systems, where it was developed by Thouless, Anderson and Palmer (TAP) to treat the Sherrington-Kirkpatrick (SK) model of disordered magnetic materials (19)

Often times, individuals working together as a team can solve hard problems beyond the capability of any individual in the team. Cooperative optimization is a newly proposed general method for attacking hard optimization problems inspired by cooperation principles in team playing. It has an established theoretical foundation and has demonstrated outstanding performances in solving real-world optimization problems. With some general settings, a cooperative optimization algorithm has a unique equilibrium and converges to it with an exponential rate regardless initial conditions and insensitive to perturbations. It also possesses a number of global optimality conditions for identifying global optima so that it can terminate its search process efficiently. This paper offers a general description of cooperative optimization, addresses a number of design issues, and presents a case study to demonstrate its power. I.

c Slow-wave sleep consists in slowly recurring waves that are associated with a large-scale spatio-temporal synchrony across neocortex. These slow-wave complexes alternate with brief episodes of fast oscillations, similar to the sustained fast oscillations that occur during the wake state. We propose that alternating fast and slow waves consolidate information acquired previously during wakefulness. Slow-wave sleep would thus begin with spindle oscillations that open molecular gates to plasticity, then proceed by iteratively ‘recalling ’ and ‘storing ’ information primed in neural assemblies. This scenario provides a biophysical mechanism consistent with the growing evidence that sleep serves to consolidate memories. © 2000 Elsevier Science B.V. All rights reserved.

Abstract. One of the greatest scientific achievements of physics in the 20th century is the discovery of quantum mechanics. The Schrödinger equation is the most fundamental equation in quantum mechanics describing the time-based evolution of the quantum state of a physical system. It has been found that the time-independent version of the equation can be derived from a general optimization algorithm. Instead of arguing for a new interpretation and possible deeper principle for quantum mechanics, this paper elaborates a few points of the equation as a general global optimization algorithm. Benchmarked against randomly generated hard optimization problems, this paper shows that the algorithm significantly outperformed a classic local optimization algorithm. The former found a solution in one second with a single trial better than the best one found by the latter around one hour after one hundred thousand trials. 1