We investigate the consequences of maximizing information transfer in a simple neural network (one input layer, one output layer), focussing on the case of non linear transfer functions. We assume that both receptive fields (synaptic efficacies) and transfer functions can be adapted to the environment. The main result is that, for bounded and invertible transfer functions, in the case of a vanishing additive output noise, and no input noise, maximization of information (Linsker'sinfomax principle) leads to a factorial code - hence to the same solution as required by the redundancy reduction principle of Barlow. We show also that this result is valid for linear, more generally unbounded, transfer functions, provided optimization is performed under an additive constraint, that is which can be written as a sum of terms, each one being specific to one output neuron. Finally we study the effect of a non zero input noise. We find that, at first order in the input noise, assumed to be small ...

In the context of both sensory coding and signal processing, building factorized codes has been shown to be an efficient strategy. In a wide variety of situations, the signal to be processed is a linear mixture of statistically independent sources. Building a factorized code is then equivalent to performing blind source separation. Thanks to the linear structure of the data, this can be done, in the language of signal processing, by finding an appropriate linear filter, or equivalently, in the language of neural modeling, by using a simple feedforward neural network. In this paper we discuss several aspects of the source separation problem. We give simple conditions on the network output which, if satisfied, guarantee that source separation has been obtained. Then we study adaptive approaches, in particular those based on redundancy reduction and maximisation of mutual information. We show how the resulting updating rules are related to the BCM theory of synaptic plasticity. Eventually...

The honeybee olfactory pathway is an attractive system for modeling: it is relatively simple, and it is well described functionally and morphologically. Moreover, due to the conservation of the olfactory structure through phylogeny, models may bring information of generic interest. From the point of view of behavior, this system has the ability of encoding the sensory messages into stable representations, and extracting key features from them. The neural bases of these mechanisms are still largely unknown; the purpose of the present paper is to present three different models of the same system, which make use of the same corpus of morphological and electrophysiological data, but which incorporate these data with different levels of details. We show the interrelations between these models and the specific contribution of each of them to the modeling of the olfactory pathway. We show that the design of the simplest model capitalized on the results of the previous ones, and that it sugges...

this paper we considered the problem of maximizing information transfer with a network of neurons made of N inputs and p outputs, focussing on the case of non linear transfer functions and arbitrary input distributions. We assumed that both the transfer functions and the synaptic efficacies could be adapted to the environment. The main consequence of our analysis is that, in the limit of small additive output noise (and an even smaller input noise), the infomax principle of Linsker implies the redundancy reduction