temporal relations requires the joint evaluation of responses from more than one neuron, only experiments that permit simultaneous measurements of responses 60528 Frankfurt from multiple units are considered. These include multi-Federal Republic of Germany electrode recordings from multiple individual cells, but also measurements of local field potentials (LFPs) and electroencephalographic (EEG) or magnetoencephalo-Most of our knowledge about the functional organization of neuronal systems is based on the analysis of the firing patterns of individual neurons that have been recorded one by one in succession. This approach permits as-sessment of event-related variations in discharge rate, but it precludes detection of any covariations in the amplitude or timing of distributed responses if these graphic (MEG) recordings. The signals of these latter

e tests of the electronic device cover the range from spontaneous activity (3--4 Hz) to stimulus-driven rates (50 Hz). Low transition probabilities can be maintained in all ranges, even though the intrinsic time constants of the device are short (# 100 ms). Synaptic transitions are triggered by elevated presynaptic rates: for low presynaptic rates, there are essentially no transitions. The synaptic device can preserve its memory for years in the absence of stimulation. Stochasticity of learning is a result of the variability of interspike intervals; noise is a feature of the distributed dynamics of the network. The fact Neural Computation 12, 2227--2258 (2000) c # 2000 Massachusetts Institute of Technology 2228 Fusi, Annunziato, Badoni, Salamon, and Amit that the synapse is binary on long timescales solves the stability problem of synaptic efficacies in the absence of stimulation. Yet stochastic learning theory

this technical difficulty by considering for example in a simplified setting only correlation variables

mutal Slovin, and Moshe Abeles. Spatiotemporal structure of millisecond time scale. cortical activity: properties and behavioral relevance. J. Neuro- The single neuron time-dependent rate function was taken physiol. 79: 2857–2874, 1998. The study was designed to reveal by many as the coding parameter (e.g., Barlow 1972, 1992, occurrences of precise firing sequences (PFSs) in cortical activity and to test their behavioral relevance. Two monkeys were trained 1994; Newsome et al. 1989; Rolls 1991). Others suggested to perform a delayed-response paradigm and to open puzzle boxes. a population coding, based on either the summed activity of Extracellular activity was recorded from neurons in premotor and neurons (Georgopoulos et al. 1986; Schwartz 1994), or the prefrontal areas with an array of six microelectrodes. An algorithm coherency in firing among cells (Eckhorn et al. 1988; Engel was developed to detect PFSs, defined as a set of three spikes and et al. 1991a–c; Gray and Singer 1992; Gray et al. 1989, two intervals with a precision of {1 ms repeating significantly 1992). Both views ignored the detailed temporal structure more than expected by chance. The expected level of repetition of cortical activity, assuming that precision is not compatible was computed based on the firing rate and the pairwise correlation with a noisy cortical environment. Despite this notion, sevof

Abstract—We present a mixed-mode analog/digital VLSI device comprising an array of leaky integrate–and–fire (I&F) neurons, adaptive synapses with spike–timing dependent plasticity, and an asynchronous event based communication infrastructure that allows the user to (re)configure networks of spiking neurons with arbitrary topologies. The asynchronous communication protocol used by the silicon neurons to transmit spikes (events) off-chip and the silicon synapses to receive spikes from the outside is based on the “address–event representation ” (AER). We describe the analog circuits designed to implement the silicon neurons and synapses and present experimental data showing the neuron’s response properties and the synapses characteristics, in response to AER input spike trains. Our results indicate that these circuits can be used in massively parallel VLSI networks of I&F neurons to simulate real–time complex spike–based learning algorithms. Index Terms—Address–event representation (AER), analog VLSI, integrate-and-fire (I&F) neurons, neuromorphic circuits, spike-based learning, spike-timing dependent plasticity (STDP). I.

prising and potentially detrimental to brain function. But another possibility is that synaptic unreliability is used by the brain for the purposes of learning (Minsky, 1954; Hinton, 1989), in analogy to the way in which unreliable genetic replication is used for evolution. Here I propose a specific implementation of this idea. According to the proposal, synapses are “hedonistic,” responding to a global reward signal by increasing their probabilities of release or failure, depending on which action immediately preceded reward. Remarkably, if each synapse in a network behaves hedonistically, selfishly seeking reward, then the network as a whole be-haves hedonistically, learning to increase its average reward by generating appropriate collective actions. This statement can be formulated and justified mathematically

Experimental evidence from a number of different preparations indicates that repeated pairing of pre- and postsynaptic action potentials can lead to long-term Brandeis University changes in synaptic efficacy, the sign and amplitude of Waltham, Massachusetts 02454-9110 which depend on relative spike timing (Levy and Steward,

Strong constraints on the neural mechanisms underlying the formation of place fields in the rodent hippocampus come from the systematic changes in spatial activity patterns that are consequent on systematic environmental manipulations. We describe an attractor network model of area CA3 in which local, recurrent, excitatory, and inhibitory interactions generate appropriate place cell representations from location- and directionspecific activity in the entorhinal cortex. In the model, familiarity with the environment, as reflected by activity in neuromodulatory systems, influences the efficacy and plasticity of the recurrent and feedforward inputs to CA3. In unfamiliar, novel, environments, mossy fiber inputs impose activity patterns on CA3, and the recurrent collaterals and the perforant path inputs are subject to graded Hebbian plasticity. The hippocampus is known to be involved in spatial learning and memory in rodents. Some of the most convincing evidence for this is the presence of place cells in areas CA3 and CA1 of the hippocampus (O’Keefe and Dostrovsky, 1971; O’Keefe, 1976) and of many other types of spatially selective cells in neighboring areas

Three different uses of a recurrent neural network (RNN) as a reservoir that is not trained but instead read out by a simple external classification layer have been described in the literature: Liquid State Machines (LSMs), Echo State Networks (ESNs) and the Backpropagation Decorrelation (BPDC) learning rule. Individual descriptions of these techniques exist, but a overview is still lacking. Here, we present a series of experimental results that compares all three implementations, and draw conclusions about the relation between a broad range of reservoir parameters and network dynamics, memory, node complexity and performance on a variety of benchmark tests with different characteristics. Next, we introduce a new measure for the reservoir dynamics based on Lyapunov exponents. Unlike previous measures in the literature, this measure is dependent on the dynamics of the reservoir in response to the inputs, and in the cases we tried, it indicates an optimal value for the global scaling of the weight matrix, irrespective of the standard measures. We also describe the Reservoir Computing Toolbox that was used for these experiments, which implements all the types of Reservoir Computing and allows the easy simulation of a wide range of reservoir topologies for a number of benchmarks.

We study analytically a model of long-term synaptic plasticity where synaptic changes are triggered by presynaptic spikes, postsynaptic spikes, and the time dierences between presynaptic and postsynaptic spikes. The changes due to correlated input and output spikes are quanti- ed by means of a learning window. We show that plasticity can lead to an intrinsic stabilization of the mean ring rate of the postsynaptic neuron. Subtractive normalization of the synaptic weights (summed over all presynaptic inputs converging on a postsynaptic neuron) follows if, in addition, the mean input rates and the mean input correlations are identical at all synapses. If the integral over the learning window is positive, ring-rate stabilization requires a non-Hebbian component, whereas such a component is not needed, if the integral of the learning window is negative. A negative integral corresponds to `anti-Hebbian' learning in a model with slowly varying ring rates. For spike-based learning, a strict distinction between Hebbian and `anti-Hebbian' rules is questionable since learning is driven by correlations on the time scale of the learning window. The correlations between presynaptic and postsynaptic ring are evaluated for a piecewise-linear Poisson model and for a noisy spiking neuron model with refractoriness. While a negative integral over the learning window leads to intrinsic rate stabilization, the positive part of the learning window picks up spatial and temporal correlations in the input.