The relative efficiency of any particular image-coding scheme should be defined only in relation to the class of images that the code is likely to encounter. To understand the representation of images by the mammalian visual system, it might therefore be useful to consider the statistics of images from the natural environment (i.e., images with trees, rocks, bushes, etc). In this study, various coding schemes are compared in relation to how they represent the information in such natural images. The coefficients of such codes are represented by arrays of mechanisms that respond to local regions of space, spatial frequency, and orientation (Gabor-like transforms). For many classes of image, such codes will not be an efficient means of representing information. However, the results obtained with six natural images suggest that the orientation and the spatial-frequency tuning of mammalian simple cells are well suited for coding the information in such images if the goal of the code is to convert higher-order redundancy (e.g., correlation between the intensities of neighboring pixels) into first-order redundancy (i.e., the response distribution of the coefficients). Such coding produces a relatively high signal-to-noise ratio and permits information to be transmitted with only a subset of the total number of cells. These results support Barlow's theory that the goal of natural vision is to represent the information in the natural environment with minimal redundancy.

In this paper, we propose a computational model of the recognition of real world scenes that bypasses the segmentation and the processing of individual objects or regions. The procedure is based on a very low dimensional representation of the scene, that we term the Spatial Envelope. We propose a set of perceptual dimensions (naturalness, openness, roughness, expansion, ruggedness) that represent the dominant spatial structure of a scene. Then, we show that these dimensions may be reliably estimated using spectral and coarsely localized information. The model generates a multidimensional space in which scenes sharing membership in semantic categories (e.g., streets, highways, coasts) are projected closed together. The performance of the spatial envelope model shows that specific information about object shape or identity is not a requirement for scene categorization and that modeling a holistic representation of the scene informs about its probable semantic category.

Abstract not found

Knill et al.

A number of researchers have suggested that in order to understand the response properties of cells in the visual pathway, we must consider the statistical structure of the natural environment. In this paper, we focus on one aspect of that structure, namely, the correlational structure which is described by the amplitude or power spectra of natural scenes. We propose that the principle insight one gains from considering the image spectra is in understanding the relative sensitivity of cells tuned to different spatial frequencies. This study employs a model in which the peak sensitivity is constant as a function of frequency with linear bandwith increasing (i.e., approximately constant in octaves). In such a model, the "response magnitude " (i.e., vector length) of cells increases as a function of their optimal (or central) spatial frequency out to about 20 cyc/deg. The result is a code in which the response to natural scenes, whose amplitude spectra typically fall as 1/f, is roughly constant out to 20 cyc/deg. An important consideration in evaluating this model of sensitivity is the fact that natural scenes show considerable variability in their amplitude spectra, with individual scenes showing falloffs which are often steeper or shallower than 1/f. Using a new measure of image structure (the "rectified contrast spectrum " or "RCS") on a set of calibrated natural images, it is shown that a large part of the variability in the spectra is due to differences in the sparseness of local

We propose that visual adaptation in orientation, spatial frequency, and motion can be understood from the perspective of optimal information transmission. The essence of the proposal is that neural response properties at the system level should be adjusted to the changing statistics of the input so as to maximize information transmission. We show that this principle accounts for several well-documented psychophysical phenomena, including the tilt aftereffect, change in contrast sensitivity and post-adaptation changes in orientation discrimination. Adaptation can also be considered on a longer time scale, in the context of tailoring response properties to natural scene statistics. From the anisotropic distribution of power in natural scenes, the proposal also predicts differences in the contrast sensitivity function across spatial frequency and orientation, including the oblique effect. 1999 Elsevier Science Ltd. All rights reserved. Keywords: Adaptation; Aftereffects; Signal-to-noise ratio; Sensitivity; Natural scenes www.elsevier.com/locate/visres 1.

Abstract not found

Abstract. A large photo collection downloaded from the internet spans a wide range of scenes, cameras, and photographers. In this paper we introduce several novel priors for statistics of such large photo collections that are independent of these factors. We then propose that properties of these factors can be recovered by examining the deviation between these statistical priors and the statistics of a slice of the overall photo collection that holds one factor constant. Specifically, we recover the radiometric properties of a particular camera model by collecting numerous images captured by it, and examining the deviation of this collection’s statistics from that of a broader photo collection whose camera-specific effects have been removed. We show that using this approach we can recover both a camera model’s non-linear response function and the spatially-varying vignetting of the camera’s different lens settings. All this is achieved using publicly available photographs, without requiring images captured under controlled conditions or physical access to the cameras. We also apply this concept to identify bad pixels on the detectors of specific camera instances. We conclude with a discussion of future applications of this general approach to other common computer vision problems. 1

We describe a formal framework for analyzing how statistical properties of natural environments and the process of natural selection interact to determine the design of perceptual and cognitive systems. The framework consists of two parts: a Bayesian ideal observer with a utility function appropriate for natural selection, and a Bayesian formulation of Darwin’s theory of natural selection. Simulations of Bayesian natural selection were found to yield new insights, for example, into the co-evolution of camouflage, color vision, and decision criteria. The Bayesian framework captures and generalizes, in a formal way, many of the important ideas of other approaches to perception and cognition.

This thesis focuses on the statistics of natural images. The first question that is to be answered is: what are natural images and why do we study them. We start with our definition, and then discuss the properties and uses of natural images. An image is a projection of an environment, and natural images are those that are taken from a natural environment, i.e., an environment that is commonly encountered by a particular organism. This means that these images represent the natural visual input (natural stimulus) of an eye. In general, images may include optical information extending over space, time (time-varying images), as well as wavelength (colour images). In this thesis, however, we restrict ourselves to images of light intensity (black and white images) that either extend exclusively over space (still images) or exclusively over time (time series). The motivation for investigating natural images is to gain a better understanding of neural processing in visual systems. Natural images and visual processing in biological systems are linked by the hypothesis that evolution has optimised visual systems to process natural stimuli. The analysis of the optimal performance of biological visual systems may inspire the building of artificial visual systems.