Many models for stereo disparity computation have been proposed, but few can be said to be truly biological. There is also a rich literature devoted to physiological studies of stereopsis. Cells sensitive to binocular disparity have been found in the visual cortex, but it is not clear whether these cells could be used to compute disparity maps from stereograms. Here we propose a model for biological stereo vision based on known receptive field profiles of binocular cells in the visual cortex and provide the first demonstration that these cells could effectively solve random dot stereograms. Our model also allows a natural integration of stereo vision and motion detection. This may help explain the existence of units tuned to both disparity and motion in the visual cortex. 1 Introduction It is well known that binocular disparity forms the basis of stereoscopic depth perception. There have been many physiological investigations on the mechanisms of stereopsis (see Freeman and Ohzawa, 19...

: 274, Introduction: 676, Discussion: 2402 Acknowledgments. Thanks to Ken Miller, Allan Dobbins, Christof Koch, and the anonymous reviewers for many helpful comments on this work. This work was funded by the National Science Foundation and the Office of Naval Research, and by a Sloan Foundation Fellowship (D.R.). Abstract Hubel and Wiesel (1962) first distinguished "simple" from "complex" cells in visual cortex, and proposed a processing hierarchy in which rows of LGN cells are pooled to drive oriented simple cell subunits, which are pooled in turn to drive complex cells. Though parsimonious and highly influential, the pure hierarchical model has since been challenged by results indicating many complex cells receive excitatory monosynaptic input from LGN cells, or do not depend on simple cell input. Alternative accounts for complex cell orientation tuning remain scant, however, and the function of monosynaptic LGN contacts onto complex cell dendrites remains unknown. We have used a ...

In order to elucidate the neural mechanisms involved in the perception of the three-dimensional (3D) orientation of a surface, we trained monkeys to discriminate the 3D orientation of a surface from binocular disparity cues using a Go/No-go type delayed-matching-tosample (DMTS) task and examined the properties of the surfaceorientation–selective (SOS) neurons. We recorded 57 SOS neurons from the caudal part of the lateral bank of the intraparietal sulcus (area CIP) of three hemispheres of two Japanese monkeys (Macaca fuscata). We tested 29 of 57 SOS neurons using the square plate of a solid figure stereogram (SFS) and random-dot stereogram (RDS) without perspective cues; almost all of the tested neurons (28/29) showed surface orientation selectivity for the SFS and/or the RDS without perspective cues. Eight of these 28 neurons (28.6%) showed selectivity for both the RDS and SFS, 7 (25.0%) were dominantly selective for the RDS, and 13 (46.4%) were dominantly selective for the SFS. These results suggest that neurons that show surface orientation tuning for the RDS without perspective cues compute surface orientation from the gradient of the binocular disparity given by the random-dot across the surface. On the other hand, neurons that show surface orientation tuning for the SFS without perspective cues may represent surface orientation primarily from the gradient of the binocular disparity along the contours. In conclusion, the SOS neurons in the area CIP are likely to operate higher order processing of disparity signals for surface perception by integrating the input signals from many disparity-sensitive neurons with different disparity tuning.

used psychophysical, physiological, and computational methods to unravel the brain's mechanisms of disparity computation. In 1960, Julez made the important contribution that stereo vision does not require monocular depth cues such as shading and perspective (see Julesz (1971)). This was demonstrated through his invention of random dot stereograms. A typical stereogram consists of two images of randomly distributed dots that are identical in all aspects except that a central square region of one image is shifted horizontally by a small distance with respect to the other image (see Fig. 6a for an example). When each image is viewed individually, it appears as nothing more than a flat field of random dots. However, when the two images are viewed dichoptically (i.e., the left and right images are presented to the left and right eyes respectively at the same time) the shifted central square region "jumps" out vividly at a different depth. This finding demonstrates that the brain can compute

Neurons in the visual cortex of monkeys respond selectively to the disparity between the images in the two eyes. Recent recordings have shown that some of the disparity-selective neurons in the primary visual cortex and the posterior parietal cortex are modulated by the distance of fixation. A population of such gain-modulated, disparity-selective neurons forms a set of basis functions of horizontal disparity and distance of fixation that can be used as an intermediate representation for computing egocentric distance. This distributed representation is consistent with psychophysical studies of human depth perception; in contrast, neurons explicitly tuned to distance are not consistent with how we perceive distance. In a population model that includes noise in the firing rates of neurons, the perceived distance is

Most models of disparity selectivity consider only the spatial properties of binocular cells. However, the temporal response is an integral component of real neurons ’ activities, and time-varying stimuli are often used in the experiments of disparity tuning. To understand the temporal dimension of V1 disparity representation, we incorporate a specific temporal response function into the disparity energy model and demonstrate that the binocular interaction of complex cells is separable into a Gabor disparity function and a positive time function. We then investigate how the model simple and complex cells respond to widely used time-varying stimuli, including motion-in-depth patterns, drifting gratings, moving bars, moving random-dot stereograms, and dynamic random-dot stereograms. It is found that both model simple and complex cells show more reliable disparity tuning to time-varying stimuli than to static stimuli, but similarities in the disparity tuning between simple and complex cells depend on the stimulus. Specifically, the disparity tuning curves of the two cell types are similar to each other for either drifting sinusoidal gratings or moving bars. In contrast, when the stimuli are dynamic random-dot stereograms, the disparity tuning of simple cells is highly variable, whereas the tuning of complex cells remains reliable. Moreover, cells with similar motion preferences in the two eyes cannot be truly tuned to motion in depth regardless of the stimulus types. These simulation results are consistent with a large body of extant physiological data, and provide some specific, testable predictions.

Figure-ground organization is a process by which the visual system identifies some image regions as fore-ground and others as background, inferring 3D layout from 2D displays. A recent study reported that edge responses of neurons in area V2 are selective for side-of-figure, suggesting that figure-ground organi-zation is encoded in the contour signals (border own-ership coding). Here, we show that area V2 combines two strategies of computation, one that exploits binocular stereoscopic information for the definition of local depth order, and another that exploits the global configuration of contours (Gestalt factors). These are combined in single neurons so that the “near ” side of the preferred 3D edge generally coincides with the preferred side-of-figure in 2D displays. Thus, area V2 represents the borders of 2D figures as edges of surfaces, as if the figures were objects in 3D space. Even in 3D displays, Gestalt factors influence the responses and can enhance or null the stereoscopic depth information.

between images presented to the two eyes induce a strong sensation of depth. More recent experiments with random-dot stereograms have shown that disparity is a sufficient cue for stereopsis (Julesz 1960, 1971). Disparity-tuned neurons in visual cortex were first demonstrated in the cat (Barlow et al. 1967; Nikara

We can see things in three dimensions because the visual system re-constructs the three-dimensional (3D) configurations of objects from their two-dimensional (2D) images projected onto the retinas. The purpose of this paper is to give an overview of the psychological background and recent physiological findings concerning three-dimensional vision. Psychophysical and computational studies have suggested that in the visual system the 3D surface orientation is first estimated independently from individual depth cues—such as binocular disparity, as well as various monocular cues including texture gradients—and then the information from these different depth cues is integrated to construct a generalized representation of the 3D surface geometry. Neurons involved in low-level disparity processing, or the detection of local absolute disparity, were found mainly in the occipital cortex, whereas neurons involved in high-level disparity processing, or the reconstruction of 3D surface orientation through the computation of disparity gradients, were found mainly in the parietal area caudal intraparietal sulcus (CIP). Neurons sensitive to texture gradients, which is one of the major monocular cues, were also found in CIP. The majority of these neurons were sensitive to disparity gradients as well, suggesting their involvement in the computation of 3D surface orientation. In CIP, neurons sensitive to multiple depth cues were widely distributed together with those sensitive to a specific depth cue, suggesting CIP’s involvement in the integration of depth information from different sources. In addition, human and monkey imaging studies have indicated convergence of multiple depth cues in CIP. These neurophysiological findings suggest that CIP plays a critical role in 3D vision

This article examines an energy model of binocular interaction with monocular and binocular response normalization. Disparity selectivity of the model neurons arises from a combination of position-shifts and phase-shifts between the monocular subfields of binocular receptive fields. Position- and phase-shifts have different quantitative properties, and it is argued that both likely contribute to the disparity selectivity of cells in V1. Modelling Binocular Neurons in the Primary Visual Cortex 25 Binocular Stimulus Phase-Difference (degrees)