The spatial receptive fields of simple cells in mammalian striate cortex have been reasonably well described physiologically and can be characterized as being localized, oriented, and ban@ass, comparable with the basis functions of wavelet transforms. Previously, we have shown that these receptive field properties may be accounted for in terms of a strategy for producing a sparse distribution of output activity in response to natural images. Here, in addition to describing this work in a more expansive fashion, we examine the neurobiological implications of sparse coding. Of particular interest is the case when the code is overcomplete--i.e., when the number of code elements is greater than the effective dimensionality of the input space. Because the basis functions are non-orthogonal and not linearly independent of each other, sparsifying the code will recruit only those basis functions necessary for representing a given input, and so the input-output function will deviate from being purely linear. These deviations from linearity provide a potential explanation for the weak forms of non-linearity observed in the response properties of cortical simple cells, and they further make predictions about the expected interactions among units in

this article we investigate to what extent the statistical properties of natural images can be used to understand the variation of receptive field properties of simple cells in the mammalian primary visual cortex. The receptive fields of simple cells have been studied extensively (e.g., Hubel & Wiesel 1968, DeValois et al. 1982a, DeAngelis et al. 1993): they are localised in space and time, have band-pass characteristics in the spatial and temporal frequency domains, are oriented, and are often sensitive to the direction of motion of a stimulus. Here we will concentrate on the spatial properties of simple cells. Several hypotheses as to the function of these cells have been proposed. As the cells preferentially respond to oriented edges or lines, they can be viewed as edge or line detectors. Their joint localisation in both the spatial domain and the spatial frequency domain has led to the suggestion that they mimic Gabor filters, minimising uncertainty in both domains (Daugman 1980, Marcelja 1980). More recently, the match between the operations performed by simple cells and the wavelet transform has attracted attention (e.g., Field 1993). The approaches based on Gabor filters and wavelets basically consider processing by the visual cortex as a general image processing strategy, relatively independent of detailed assumptions about image statistics. On the other hand, the edge and line detector hypothesis is based on the intuitive notion that edges and lines are both abundant and important in images. This theme of relating simple cell properties with the statistics of natural images was explored extensively by Field (1987, 1994). He proposed that the cells are optimized specifically for coding natural images. He argued that one possibility for such a code, sparse coding...

Introduction It has been suggested that visual representations are optimised to transmit the maximum information about the images encountered in everyday life (Uttley, 1973; Linsker, 1988; Barlow, 1989). This simple assumption has proven sufficient to account for the characteristics of large monopolar cells in the fly (Srinivasan et al., 1982; Hateren, 1992; Laughlin, 1981), the temporal characteristics of retinal ganglion cells (Dong & Atick, 1995), human spatial frequency thresholds (Atick & Redlich, 1992; Van Hateren, 1993), and the psychophysics of orientation perception for short presentation times (Baddeley & Hancock, 1991). Maximisation of information is a powerful theoretical principle that leads to testable predictions about the firing patterns of neurons. However, to generate specific predictions we must make some assumptions about the nature of the neural code and the type of constraint that limits its information carrying capacity. To appl

second precision on subsequent generate highly variable spike trains because they re- stimulus repeats. Here we develop a mathematical ceive large numbers of unsynchronized synaptic inputs description of the firing process that, given the recent and because many of these are not under experimental visual input, accurately predicts the timing of individ- control. By contrast, neurons in the early visual system--- ual spikes. The formalism is successful in matching the from the retina to the lateral geniculate nucleus (LGN) spike trains from retinal ganglion cells in salamander, to area V1---can deliver remarkably reproducible spike rabbit, and cat, as well as from lateral geniculate nu- trains, whose trial-to-trial variability is clearly lower than cleus neurons in cat. It adapts to many different re- predicted from the simple firing rate formalism (Berry et sponse typ

critical test of our understanding of sensory coding, however, is to take an opposite approach: to reconstruct sensory inputs from recorded neuronal responses. The decoding approach can provide an objective assessment of what and how much information is available in the neuronal responses. Although the f unction of the brain is not necessarily to reconstruct sensory inputs faithfully, these studies may lead to new insights into the f unctions of neuronal circuits in sensory processing (Rieke et al., 1997). The decoding approach has been used to study several sensory systems (Bialek et al., 1991; Theunissen and Miller, 1991; Rieke et al., 1993, 1997; Roddey and Jacobs, 1996; Warland et al., 1997; Dan et al., 1998). Most of these studies aimed to reconstruct temporal signals from the response of a single neuron (Bialek et al., 1991; Rieke et al., 1993, 1995; Roddey and Jacobs, 1996) or a small number of neurons (Warland et al., 1997). An important challenge in understanding the mammalia

The performance of unsupervised learning models for natural images is evaluated quantitatively by means of information theory. We estimate the gain in statistical independence (the multi-information reduction) achieved with independent component analysis (ICA), principal component analysis (PCA), zero-phase whitening, and predictive coding. Predictive coding is translated into the transform coding framework, where it can be characterized by the constraint of a triangular filter matrix. A randomly sampled whitening basis and the Haar wavelet are included into the comparison as well. The comparison of all these methods is carried out for different patch sizes, ranging from 2x2 to 16x16 pixels. In spite of large differences in the shape of the basis functions, we find only small differences in the multi-information between all decorrelation transforms (5% or less) for all patch sizes. Among the second-order methods, PCA is optimal for small patch sizes and predictive coding performs best for large patch sizes. The extra gain achieved with ICA is always less than 2%. In conclusion, the `edge filters‘ found with ICA lead only to a surprisingly small improvement in terms of its actual objective.

A major problem a visual system faces is how to fit the large intensity variation of natural image streams into the limited dynamic range of its neurons. One of the means to accomplish this is through the use of gain control. In order to investigate this, natural time series of intensities were measured, as well as the responses of blowfly photoreceptors and Large Monopolar Cells (LMCs) to these time series. Time series representative of what each photoreceptor of a real visual system would normally receive were measured with an optical system measuring the light intensity of a spot comparable to the field of view of single human foveal cones. This system was worn on a head-band by a freely walking person. Resulting time series have rms-contrasts ranging from an average of 0.45 for 1 second segments to 1.39 for 100 second segments (both when limited to frequencies up to 100 Hz). Power spectra behave approximately as 1/f (f: temporal frequency). Measured time series were subsequently presented to fly photoreceptors and LMCs by playing them back on an LED. The results show that fast gain controls indeed keep the response within the dynamic range of the cells and that a large part of this range is actually used for packing the information in natural time series.

The responses of simple cells in primary visual cortex to sinusoidal gratings can primarily be predicted from their spatial receptive fields, as mapped using spots or bars. Although this quasilinearity is well documented, it is not clear whether it holds for complex natural stimuli. We recorded from simple cells in the primary visual cortex of anesthetized ferrets while stimulating with flashed digitized photographs of natural scenes. We applied standard reverse-correlation methods to quantify the average natural stimulus that invokes a neuronal response. Although these maps cannot be the receptive fields, we find that they still predict the preferred orientation of grating for each cell very well (r � 0.91); they do not predict the spatial-frequency tuning. Using a novel application of the linear reconstruction method called regularized pseudoinverse, we were able to recover high-resolution receptive-field maps from the responses to a relatively small number of natural scenes. These receptive-field maps not only predict the optimum orientation of each cell (r � 0.96) but also the spatial-frequency optimum (r � 0.89); the maps also predict the tuning bandwidths of many cells. Therefore, our first conclusion is that the tuning preferences of the cells are primarily linear and constant across stimulus type. However, when we used these maps to predict the actual responses of the cells to natural scenes, we did find evidence of expansive output nonlinearity and nonlinear influences from outside the classical receptive fields, orientation tuning, and spatial-frequency tuning.

A central hypothesis concerning sensory processing is that the neuronal circuits are specifically adapted to represent natural stimuli efficiently. Here we show a novel effect in cortical coding of natural images. Using spike-triggered average or spike-triggered covariance analyses, we first identified the visual features selectively represented by each cortical neuron from its responses to natural images. We then measured the neuronal sensitivity to these features when they were present in either natural images or random stimuli. We found that in the responses of complex cells, but not of simple cells, the sensitivity was markedly higher for natural images than for random stimuli. Such elevated sensitivity leads to increased detectability of the visual features and thus an improved cortical representation of natural scenes. Interestingly, this effect is due not to the spatial power spectra of natural images, but to their phase regularities. These results point to a distinct visual-coding strategy that is mediated by contextual modulation of cortical responses tuned to the spatial-phase structure of natural scenes.

This thesis focuses on the statistics of natural images. The first question that is to be answered is: what are natural images and why do we study them. We start with our definition, and then discuss the properties and uses of natural images. An image is a projection of an environment, and natural images are those that are taken from a natural environment, i.e., an environment that is commonly encountered by a particular organism. This means that these images represent the natural visual input (natural stimulus) of an eye. In general, images may include optical information extending over space, time (time-varying images), as well as wavelength (colour images). In this thesis, however, we restrict ourselves to images of light intensity (black and white images) that either extend exclusively over space (still images) or exclusively over time (time series). The motivation for investigating natural images is to gain a better understanding of neural processing in visual systems. Natural images and visual processing in biological systems are linked by the hypothesis that evolution has optimised visual systems to process natural stimuli. The analysis of the optimal performance of biological visual systems may inspire the building of artificial visual systems.