The effectiveness of various stimulus identification (decoding) procedures for extracting the information carried by the responses of a population of neurons to a set of repeatedly presented stimuli is studied analytically, in the limit of short time windows. It is shown that in this limit, the entire information content of the responses can sometimes be decoded, and when this is not the case, the lost information is quantified. In particular, the mutual information extracted by taking into account only the most likely stimulus in each trial turns out to be, if not equal, much closer to the true value than that calculated from all the probabilities that each of the possible stimuli in the set was the actual one. The relation between the mutual information extracted by decoding and the percentage of correct stimulus decodings is also derived analytically in the same limit, showing that the metric content index can be estimated reliably from a few cells recorded from brief periods. Computer simulations as well as the activity of real neurons recorded in the primate hippocampus serve to confirm these results and illustrate the utility and limitations of the approach.

The way groups of auditory neurons interact to code acoustic information is investigated using an information theoretic approach. Identifying the case of stimulus-conditioned independent neurons, we develop redundancy measures that allow enhanced information estimation for groups of neurons. These measures are then applied to study the collaborative coding efficiency in two processing stations in the auditory pathway: the inferior colliculus (IC) and the primary auditory cortex (A1). Under two different coding paradigms we show differences in both information content and group redundancies between IC and cortical auditory neurons. These results provide for the first time a direct evidence for redundancy reduction along the ascending auditory pathway, as has been hypothesized by Barlow (1959). The redundancy effects under the single-spikes coding paradigm are significant only for groups

The overarching purpose of the studies presented in this report is the exploration of the uses of information theory and Bayesian inference applied to neural codes. Two approaches were taken: Starting from first principles, a coding mechanism is proposed, the results are compared to a biological neural code. Secondly, tools from information theory are used to measure the information contained in a biological neural code. Chapter 3: The REC model proposed by Harpur and Prager [33] codes inputs into a sparse, factorial representation, maintaining reconstruction accuracy. Here I propose a modification of the REC model to determine the optimal network dimensionality. The resulting code for unfiltered natural images is accurate, highly sparse and a large fraction of the code elements show localized features. Furthermore, I propose an activation algorithm for the network that is faster and more accurate than a gradient descent based activation method. Moreover, it is demonstrated that asymmetric noise promotes sparseness. Chapter 4: A fast, exact alternative to Bayesian classification is introduced. Computational time is quadratic in both the number of observed data points and the number of degrees of freedom of the underlying model. As an example application, responses of single neurons from high-level visual cortex (area STSa) to rapid sequences of complex visual stimuli are analyzed. Chapter 5: I present an exact Bayesian treatment of a simple, yet sufficiently general probability distribution model. The model complexity, exact values of the expectations of entropies and their variances can be computed with polynomial effort given the data. The expectation of the mutual information becomes thus available, too, and a strict upper bound on its variance. The resulting algorithm is first tested on artificial data. To that end, an information theoretic similarity measure is derived. Second, the algorithm is demonstrated to be useful in neuroscience by studying the information content of the neural responses analyzed in the previous chapter. It is shown that the information throughput of STS neurons is maximized for stimulus durations ≈ 60ms.

y to describe the main, usual form (or forms) of communication. We should take the approach of the moderately bright investigator, and leave the discovery of exceptional facts for later on. Further, we should try to quantify how much is communicated in each situation, because only a quantitative comparison allows to assess different codes, especially if they share part of the content of what is being communicated. Information theory [1] has been developed precisely to quantify communication, and is therefore quintessential to an appraisal of neural codes. Applying information theory to neural activity (rather than to the synthetic communication systems for which it was developed) is however riddled with practical problems and subtleties, which must be clarified before reporting experimental results. In this chapter, we do not consider other means of neuronal communication than the emission of action potentials, or spikes, and regard them as selfsimilar all-or-none even

The present study introduces information-geometric measures to analyze neural ring patterns by taking not only the second-order but also higher-order interactions among neurons into ac-count. Information geometry provides useful tools and concepts for this purpose, including the orthogonality of coordinate pa-rameters and the Pythagoras relation in the Kullback-Leibler di-vergence. Based on this orthogonality, we show anovel method to analyze spike ring patterns by decomposing the interactions of neurons of various orders. As a result, purely pairwise, triple-wise, and higher-order interactions are singled out. We also demonstrate the bene ts of our proposal by using real neural data, recorded in the prefrontal and parietal cortices of mon-keys. 1

The way groups of auditory neurons interact to code acoustic information is investigated using an information theoretic approach. Identifying the case of stimulus-conditioned independent neurons, we develop redundancy measures that allow enhanced information estimation for groups of neurons. These measures are then applied to study the collaborative coding eciency in two processing stations in the auditory pathway: the inferior colliculus (IC) and the primary auditory cortex (A1). Under two dierent coding paradigms we show dierences in both information content and group redundancies between IC and cortical auditory neurons. These results provide for the rst time a direct evidence for redundancy reduction along the ascending auditory pathway, as has been hypothesized by Barlow (1959). The redundancy eects under the single-spikes coding paradigm are signicant only for groups larger than ten cells, and cannot be revealed with the standard redundancy measures that use only pairs of cells. Our results suggest that redundancy reduction transformations are not limited to low level sensory processing (aimed to reduce redundancy in input statistics) but are applied even at cortical sensory stations. 1

This document summarizes the examples that illuminate the merits and applicability of the method proposed in (Nakahara and Amari, accepted), using artificially simulated data as well as experimental data from the prefrontal and dorsal extrastriate visual cortices of monkeys (Anderson et al., 1999). Some examples are the same as in (Nakahara and Amari, accepted), while new examples, particularly including experimental data and the case of auto-correlation, are included. Thus, this document should be read as a companion of (Nakahara and Amari, accepted).