In order to elucidate the neural mechanisms involved in the perception of the three-dimensional (3D) orientation of a surface, we trained monkeys to discriminate the 3D orientation of a surface from binocular disparity cues using a Go/No-go type delayed-matching-tosample (DMTS) task and examined the properties of the surfaceorientation–selective (SOS) neurons. We recorded 57 SOS neurons from the caudal part of the lateral bank of the intraparietal sulcus (area CIP) of three hemispheres of two Japanese monkeys (Macaca fuscata). We tested 29 of 57 SOS neurons using the square plate of a solid figure stereogram (SFS) and random-dot stereogram (RDS) without perspective cues; almost all of the tested neurons (28/29) showed surface orientation selectivity for the SFS and/or the RDS without perspective cues. Eight of these 28 neurons (28.6%) showed selectivity for both the RDS and SFS, 7 (25.0%) were dominantly selective for the RDS, and 13 (46.4%) were dominantly selective for the SFS. These results suggest that neurons that show surface orientation tuning for the RDS without perspective cues compute surface orientation from the gradient of the binocular disparity given by the random-dot across the surface. On the other hand, neurons that show surface orientation tuning for the SFS without perspective cues may represent surface orientation primarily from the gradient of the binocular disparity along the contours. In conclusion, the SOS neurons in the area CIP are likely to operate higher order processing of disparity signals for surface perception by integrating the input signals from many disparity-sensitive neurons with different disparity tuning.

Neurons in the visual cortex of monkeys respond selectively to the disparity between the images in the two eyes. Recent recordings have shown that some of the disparity-selective neurons in the primary visual cortex and the posterior parietal cortex are modulated by the distance of fixation. A population of such gain-modulated, disparity-selective neurons forms a set of basis functions of horizontal disparity and distance of fixation that can be used as an intermediate representation for computing egocentric distance. This distributed representation is consistent with psychophysical studies of human depth perception; in contrast, neurons explicitly tuned to distance are not consistent with how we perceive distance. In a population model that includes noise in the firing rates of neurons, the perceived distance is

Most models of disparity selectivity consider only the spatial properties of binocular cells. However, the temporal response is an integral component of real neurons ’ activities, and time-varying stimuli are often used in the experiments of disparity tuning. To understand the temporal dimension of V1 disparity representation, we incorporate a specific temporal response function into the disparity energy model and demonstrate that the binocular interaction of complex cells is separable into a Gabor disparity function and a positive time function. We then investigate how the model simple and complex cells respond to widely used time-varying stimuli, including motion-in-depth patterns, drifting gratings, moving bars, moving random-dot stereograms, and dynamic random-dot stereograms. It is found that both model simple and complex cells show more reliable disparity tuning to time-varying stimuli than to static stimuli, but similarities in the disparity tuning between simple and complex cells depend on the stimulus. Specifically, the disparity tuning curves of the two cell types are similar to each other for either drifting sinusoidal gratings or moving bars. In contrast, when the stimuli are dynamic random-dot stereograms, the disparity tuning of simple cells is highly variable, whereas the tuning of complex cells remains reliable. Moreover, cells with similar motion preferences in the two eyes cannot be truly tuned to motion in depth regardless of the stimulus types. These simulation results are consistent with a large body of extant physiological data, and provide some specific, testable predictions.

Abstract — The role of behavior for the acquisition of sensory representations has been underestimated in the past. We study this question for the task of learning vergence eye movements allowing proper fixation of objects. We model the development of this skill with an artificial neural network based on reinforcement learning. A biologically plausible reward mechanism that is responsible for driving behavior and learning of the representation of disparity is proposed. The network learns to perform vergence eye movements between natural images of objects by receiving a reward whenever an object is fixated with both eyes. Disparity tuned neurons emerge robustly in the hidden layer during development. The characteristics of the cells ’ tuning curves depend strongly on the task: if mostly small vergence movements are to be performed, tuning curves become narrower at small disparities, as has been measured experimentally in barn owls. Extensive training to discriminate between small disparities leads to an effective enhancement of sensitivity of the tuning curves. Index Terms — disparity tuning, neural network, vergence, reinforcement learning, natural images

The visual brain consists of several parallel, functionally specialized processing systems, each having several stages (nodes) which terminate their tasks at different times; consequently, simultaneously presented attributes are perceived at the same time if processed at the same node and at different times if processed by different nodes. Clinical evidence shows that these processing systems can act fairly autonomously. Damage restricted to one system compromises specifically the perception of the attribute that that system is specialized for; damage to a given node of a processing system that leaves earlier nodes intact results in a degraded perceptual capacity for the relevant attribute, which is directly related to the physiological capacities of the cells left intact by the damage. By contrast, a system that is spared when all others are damaged can function more or less normally. Moreover, internally created visual percepts—illusions, afterimages, imagery, and hallucinations—activate specifically the nodes specialized for the attribute perceived. Finally, anatomical evidence shows that there is no final integrator station in the brain, one which receives input from all visual areas; instead, each node has multiple outputs and no node is recipient only. Taken together, the above evidence leads us to propose that each node of a processing-perceptual system creates its own microconsciousness. We propose that, if any binding occurs to give us our integrated image of the visual world, it must be a binding between microconsciousnesses generated at different nodes. Since any two microconsciousnesses generated at any two nodes can be bound together, perceptual integration is not hierarchical, but parallel and postconscious. By contrast, the neural machinery conferring properties on those cells whose activity has a conscious correlate is hierarchical, and we refer to it as generative binding, to distinguish it from the binding that might occur between the microconsciousnesses. © 1999 Academic Press

between images presented to the two eyes induce a strong sensation of depth. More recent experiments with random-dot stereograms have shown that disparity is a sufficient cue for stereopsis (Julesz 1960, 1971). Disparity-tuned neurons in visual cortex were first demonstrated in the cat (Barlow et al. 1967; Nikara

This paper addresses whether the signals from binocular cells in cortical area V1 are accurate enough to explain the precision of stereoacuity. Previous work (Cumming and Parker, 1997, 1999) has suggested that V1 is not the cortical site at which the properties of stereoscopic depth perception are f ully realized. However, one might expect that the properties of binocular neurons in V1 would limit stereo performance in much the same way that the properties of the retina constrain spatial visual acuity. There is an alternative view: for example, Poggio and Poggio (1984) observed that "the threshold of stereoacuity is more than one order of magnitude smaller than the width of tuning of disparity-sensitive cells." This lays down two challenges. The first is to see whether evidence for better disparity sensitivity can be obtained, bringing the neural processing of disparity in line with the other properties cited above. The second is to discover whether cells with the highest sensitivities differ in some characteristic way from other neurons within the primary visual cortex

We can see things in three dimensions because the visual system re-constructs the three-dimensional (3D) configurations of objects from their two-dimensional (2D) images projected onto the retinas. The purpose of this paper is to give an overview of the psychological background and recent physiological findings concerning three-dimensional vision. Psychophysical and computational studies have suggested that in the visual system the 3D surface orientation is first estimated independently from individual depth cues—such as binocular disparity, as well as various monocular cues including texture gradients—and then the information from these different depth cues is integrated to construct a generalized representation of the 3D surface geometry. Neurons involved in low-level disparity processing, or the detection of local absolute disparity, were found mainly in the occipital cortex, whereas neurons involved in high-level disparity processing, or the reconstruction of 3D surface orientation through the computation of disparity gradients, were found mainly in the parietal area caudal intraparietal sulcus (CIP). Neurons sensitive to texture gradients, which is one of the major monocular cues, were also found in CIP. The majority of these neurons were sensitive to disparity gradients as well, suggesting their involvement in the computation of 3D surface orientation. In CIP, neurons sensitive to multiple depth cues were widely distributed together with those sensitive to a specific depth cue, suggesting CIP’s involvement in the integration of depth information from different sources. In addition, human and monkey imaging studies have indicated convergence of multiple depth cues in CIP. These neurophysiological findings suggest that CIP plays a critical role in 3D vision

We measured the ability to fuse dichoptic images of a horizontal line alone or in the presence of a textured background with different vertical disparity. Nonius-line measurements of vertical vergence were also obtained. Diplopia thresholds and vertical vergence gains were much higher in response to an isolated vertically disparate line than to one with a zero vertical-disparity background. The effect of the background was maximum when it was coplanar with the target and decreased with increasing relative horizontal disparity. We conclude that vertical disparities are integrated over a restricted range of horizontal disparities to drive vertical vergence. 2000 Elsevier Science Ltd. All rights reserved.

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : xiii I Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 A. Spatial representations and sensori-motor coordination : : : : : : : : : 1 B. The posterior parietal cortex : : : : : : : : : : : : : : : : : : : : : : : 2 C. Neural code for spatial representations : : : : : : : : : : : : : : : : : : 4 1. Dynamic remapping : : : : : : : : : : : : : : : : : : : : : : : : : : 4 2. Gain modulation : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6 3. The Zipser and Andersen Network : : : : : : : : : : : : : : : : : : 6 D. Parallel vectorial representations : : : : : : : : : : : : : : : : : : : : : 9 E. Thesis Outline : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10 1. Hierarchy in spatial representations : : : : : : : : : : : : : : : : : 10 2. A basis function approach for spatial representation : : : : : : : : 11 II Egocentric spatial representation in early vision : :...