Harvard University. The work toward attaining "artificial intelligence’ ’ is the center of considerable computer research, design, and application. The field is in its starting transient, characterized by many varied and independent efforts. Marvin Minsky has been requested to draw this work together into a coherent summary, supplement it with appropriate explanatory or theoretical noncomputer information, and introduce his assessment of the state of the art. This paper emphasizes the class of activities in which a general-purpose computer, complete with a library of basic programs, is further programmed to perform operations leading to ever higher-level information processing functions such as learning and problem solving. This informative article will be of real interest to both the general Proceedings reader and the computer specialist.-- The Guest Editor.

Models of unsupervised correlation-based (Hebbian) synaptic plasticity are typically unstable: either all synapses grow until each reaches the maximum allowed strength, or all synapses decay to zero strength. A common method of avoiding these outcomes is to use a constraint that conserves or limits the total synaptic strength over a cell. We study the dynamical effects of such constraints. Two methods of enforcing a constraint are distinguished, multiplicative and subtractive. For otherwise linear learning rules, multiplicative enforcement of a constraint results in dynamics that converge to the principal eigenvector of the operator determining unconstrained synaptic development. Subtractive enforcement, in contrast, typically leads to a final state in which almost all synaptic strengths reach either the maximum or minimum allowed value. This final state is often dominated by weight configurations other than the principal eigenvector of the unconstrained operator. Multiplica...

Abstract not found

this paper is to present such a reformulation. The cell assembly provides the cognitive system with flexibility far beyond the simple activation of concepts. Instead of viewing the assembly as simply active or latent we see the activation of the assembly as coming in a series of phases. Each phase of activity serves a different purpose, giving the theory the power and flexibility to handle a wide range of psychological data.

minicolumns, forgetting, incremental learning, reaction time Summary: During the last few decades we have seen a convergence among ideas and hypotheses regarding functional principles underlying human memory. Hebb’s now more than fifty years old conjecture concerning synaptic plasticity and cell assemblies, formalized mathematically as attractor neural networks, has remained among the most viable and productive theoretical frameworks. It suggests plausible explanations for Gestalt aspects of active memory like perceptual completion, reconstruction and rivalry. We review the biological plausibility of these theories and discuss some critical issues concerning their associative memory functionality in the light of simulation studies of models with palimpsest memory properties. The focus is on memory properties and dynamics of networks modularized in terms of cortical minicolumns and hypercolumns. Biophysical compartmental models demonstrate attractor dynamics that support cell assembly operations with fast convergence and low firing rates. Using a scaling model we obtain reasonable relative connection densities and amplitudes. An abstract attractor network model reproduces systems level psychological phenomena seen in human memory experiments as the Sternberg and von Restorff effects. We conclude that there is today considerable substance in Hebb's theory of cell assemblies and its attractor network formulations, and that they have contributed to increasing our understanding of cortical associative memory function. The criticism raised with regard to biological and psychological plausibility as well as low storage capacity, slow retrieval etc has largely been disproved. Rather, this paradigm has gained further support from new experimental data as well as computational modeling.

Abstract—Training multilayer neural networks is typically carried out using descent techniques such as the gradient-based backpropagation (BP) of error or the quasi-Newton approaches including the Levenberg–Marquardt algorithm. This is basically due to the fact that there are no analytical methods to find the optimal weights, so iterative local or global optimization techniques are necessary. The success of iterative optimization procedures is strictly dependent on the initial conditions, therefore, in this paper, we devise a principled novel method of backpropagating the desired response through the layers of a multilayer perceptron (MLP), which enables us to accurately initialize these neural networks in the minimum mean-square-error sense, using the analytic linear least squares solution. The generated solution can be used as an initial condition to standard iterative optimization algorithms. However, simulations demonstrate that in most cases, the performance achieved through the proposed initialization scheme leaves little room for further improvement in the mean-square-error (MSE) over the training set. In addition, the performance of the network optimized with the proposed approach also generalizes well to testing data. A rigorous derivation of the initialization algorithm is presented and its high performance is verified with a number of benchmark training problems including chaotic time-series prediction, classification, and nonlinear system identification with MLPs. Index Terms—Approximate least-squares training of multilayer perceptrons (MLPs), backpropagation (BP) of desired response, neural network initialization. I.

Insights into brain’s high-level computational principles will lead to novel cognitive systems, computing architectures, programming paradigms, and numerous practical applications. An important step towards this end is the study of large networks of cortical spiking neurons. We have built a cortical simulator, C2, incorporating several algorithmic enhancements to optimize the simulation scale and time, through: computationally efficient simulation of neurons in a clock-driven and synapses in an event-driven fashion; memory efficient representation of simulation state; and communication efficient message exchanges. Using phenomenological, single-compartment models of spiking neurons and synapses with spike-timing dependent plasticity, we represented a rat-scale cortical model (55 million neurons, 442 billion synapses) in 8TB memory of a 32,768processor BlueGene/L. With 1 millisecond resolution for neuronal dynamics and 1-20 milliseconds axonal delays, C2 can simulate 1 second of model time in 9 seconds per Hertz of average neuronal firing rate. In summary, by combining state-of-the-art hardware with innovative algorithms and software design, we simultaneously achieved unprecedented time-to-solution on an unprecedented problem size. 1.

this paper, our goals are to introduce and to discuss these issues. We argue for an essentially utilitarian view of computational modeling. We suggest that the main function of computational modeling is to support an interactive process of "probing and prediction" through which models can be interacted with in a way that provides both guidance for empirical research and also sufficient depth to support interactive modification of the underlying theory. We propose that models, just as the systems they are models of, can only be understood (and evaluated) with respect to a given level of description and a specific set of criteria associated with that level. We also claim that models gain explanatory power as well as practical usefulness when they are emergent, that is, when they provide an account of how the principles of organization at a given level of description constrain and define structure at a higher level of description. For this reason, connectionist models appear to provide the most fruitful modeling framework today.

this paper is to solve equation 2.2. In the radially symmetric case we know that the eigenstates have the form /(r) / e

RF-LISSOM, a self-organizing model of laterally connected orientation maps in the primary visual cortex, was used to study the psychological phenomenon known as the tilt aftereffect. The same self-organizing processes that are responsible for the long-term development of the map and its lateral connections are shown to result in tilt aftereffects over short time scales in the adult. The model allows observing large numbers of neurons and connections simultaneously, making it possible to relate higher-level phenomenato low-level events, which is difficult to do experimentally. The results give computational support for the idea that direct tilt aftereffects arise from adaptive lateral interactions between feature detectors, as has long been surmised. They also suggest that indirect effects could result from the conservation of synaptic resourcesduring this process. The model thus provides a unified computational explanation of self-organization and both direct and indirect tilt aftereff...