The benefits of using Intrinsic Plasticity (IP), an unsupervised, local, biologically inspired adaptation rule that tunes the probability density of a neuron’s output towards an exponential distribution – thereby realizing an information maximization – have already been demonstrated. In this work, we extend the ideas of this adaptation method to a more commonly used nonlinearity and a Gaussian output distribution. After deriving the learning rules, we show the effects of the bounded output of the transfer function on the moments of the actual output distribution. This allows us to show that the rule converges to the expected distributions, even in random recurrent networks. The IP rule is evaluated in a Reservoir Computing setting, which is a temporal processing technique which uses random, un-trained recurrent networks as excitable media, where the network’s state is fed to a linear regressor used to calculate the desired output. We present an experimental comparison of the different IP rules on three benchmark tasks with different characteristics. Furthermore, we show that this unsupervised reservoir adaptation is able to adapt networks with very constrained topologies, such as a 1D lattice which generally shows quite unsuitable dynamic behavior, to a reservoir that can be used to solve complex tasks. We clearly demonstrate that IP is able to make Reservoir Computing more robust: the internal dynamics can autonomously tune themselves – irrespective of initial weights or input scaling – to the dynamic regime which is optimal for a given task.

Abstract. Spiking Neuron Networks (SNNs) are often referred to as the 3 rd generation of neural networks. They derive their strength and interest from an accurate modelling of synaptic interactions between neurons, taking into account the time of spike emission. SNNs overcome the computational power of neural networks made of threshold or sigmoidal units. Based on dynamic event-driven processing, they open up new horizons for developping models with an exponential capacity of memorizing and a strong ability to fast adaptation. Today, the main challenge is to discover efficient learning rules that might take advantage of the specific features of SNNs while keeping the nice properties (general-purpose, easy-to-use, available simulators, etc.) of current connectionist models (such as MLP, RBF or SVM). The present survey relates the history of the “spiking neuron ” and summarizes the most currenlty in use models of neurons and networks, in Section 1. The computational power of SNNs is addressed in Section 2 and the problem of learning in networks of spiking neurons is tackled in Section 3, with insights into the tracks currently explored for solving it. Section 4 reviews the tricks of implementation and discuss several simulation frameworks. Examples of application domains are proposed in Section 5, mainly in speech processing and computer vision, emphasizing the temporal aspect of pattern recognition by SNNs.

Abstract—In this paper it is shown that Reservoir Computing can be successfully applied to perform real-time detection of epileptic seizures in Electroencephalograms (EEGs). Absence and tonic-clonic seizures are detected on intracranial EEG coming from rats. This resulted in an area under the Receiver Operating Characteristics (ROC) curve of more than 0.99 on the data that was used. For absences an average detection delay of 0.3s was noted, for tonic-clonic seizures this was 1.5s. Since it was possible to process 15h of data on an average computer in 14.5 minutes all conditions are met for a fast and reliable real-time detection system. Index Terms—epilepsy, real-time seizure detection, Reservoir Computing, absences, SWDs, tonic-clonic seizures, neural networks,

Automatic speech recognition has gradually improved over the years, but the reliable recognition of unconstrained speech is still not within reach. In order to achieve a breakthrough, many research groups are now investigating new methodologies that have potential to outperform the Hidden Markov Model technology that is at the core of all present commercial systems. In this paper, it is shown that the recently introduced concept of Reservoir Computing might form the basis of such a methodology. In a limited amount of time, a reservoir system that can recognize the elementary sounds of continuous speech has been built. The system already achieves a state-of-the-art performance, and there is evidence that the margin for further improvements is still significant. 1