Various visual cues provide information about depth and shape in a scene. When several of these cues are simultaneously available in a single location in the scene, the visual system attempts to combine them. In this paper, we discuss three key issues relevant to the experimental analysis of depth cue combination in human vision: cue promotion, dynamic weighting of cues, and robustness of cue combination. We review recent psychophysical studies of human depth cue combination in light of these issues. We organize the discussion and review as the development of a model of the depth cue combination process termed modified weak fusion (MWF). We relate the MWF framework to Bayesian theories of cue combination. We argue that the MWF model is consistent with previous experimental results and is a parsimonious summary of these results. While the MWF model is motivated by normative considerations, it is primarily intended to guide experimental analysis of depth cue combination in human vision. We describe experimental methods, analogous to perturbation analysis, that permit us to analyze depth cue combination in novel ways. In particular these methods allow us to investigate the key issues we have raised. We summarize recent experimental tests of the MWF framework that use these methods. Depth Multiple cues Sensor fusion

We examined how depth information from two different cue types (object motion and texture gradient) is integrated into a single estimate in human vision. Two critical assumptions of a recent model of depth cue combination (termed modified weak fusion) were tested. The first assumption is that the overall depth estimate is a weighted linear combination of the estimates derived from the individual cues, after initial processing needed to bring them to a common format. The second assumption is that the weight assigned to a cue reflects the apparent reliability of that cue in a particular scene. By this account, the depth combination rule is linear and dynamic, changing in a predictable fashion in response to the particular scene and viewing conditions. A novel procedure was used to measure the weights assigned to the texture and motion cues across experimental conditions. This procedure uses a type of perturbation analysis. The results are consistent with the weighted linear combination rule. In addition, when either cue is corrupted by added noise, the weighted linear combination rule shifts in favor of the uncontaminated cue.

An optimal linear system for integrating visual cues to 3D surface geometry weights cues in inverse proportion to their uncertainty. The problem of integrating texture and stereo information for judgments of planar surface slant provides a strong test of optimality in human perception. Since the accuracy of slant from texture judgments changes by an order of magnitude from low to high slants, optimality predicts corresponding changes in cue weights as a function of surface slant. Furthermore, since humans show significant individual differences in their abilities to use both texture and stereo information for judgments of 3D surface geometry, the problem admits the stronger test that individual differences in subjectsÕ thresholds for discriminating slant from the individual cues should predict individual differences in cue weights. We tested both predictions by measuring slant discrimination thresholds and stereo/texture cue weights as a function of surface slant for multiple subjects. The results bear out both predictions of optimality, with the exception of an apparent slight under-weighting of texture information. This may be accounted for by factors specific to the stimuli used to isolate stereo information in the experiments. Taken together, the results are consistent with the hypothesis that humans optimally combine the two cues to surface slant, with cue weights proportional to the subjective reliability of the cues.

We test hypotheses concerning human cue combination in a slant estimation task. Observers

depth ambiguities. Without promoting the cues, their raw data (e.g., disparities and velocities) are in different units so that simple cue-combination strategies, such as averaging the depth estimates made using each cue, are impossible. When the missing parameters are the eye positions (vergence, gaze directions, and torsions), the promotion process is referred to as depth scaling. In particular, in central gaze, the raw sensory data for the cue (velocities, disparities, etc.) are scaled by (that is, multiplied by, or multiplied by the square of) an estimate of the fixation distance. To the extent that this scaling is done accurately, the result is depth constancy: perceived depth that is independent of changes in viewing conditions. In this hapter we will limit our discussion of cue promotion to the issue of scaling by the fixation distance. We review a number of ways in which depth scaling may be accomplished. Micha

In three experiments, we investigated the integration of three-dimensional information provided over time by different depth cues. In the first experiment, we found that the perceptual derivation of surface orientation from the optic flow was affected by the prior presentation of static stereo information in the same spatial location. This bias weakened as the length of the motion sequence increased, but it was still present after 800 msec. In the second experiment, conversely, we found that the perceived orientation of a stereo-specified surface was not influenced by the prior presentation of a static stereo surface. In a third experiment, we found that two surfaces defined by identical disparity fields did not elicit the same perceived depth if, previously, one of them had been specified by a conjunction of stereo and motion information. This effect was found to last for at least 400 msec. Taken together, these findings indicate that interactions exist among different sources of depth information, even when they are provided at different moments of time. Human observers obtain information about threedimensional (3-D) shape from a large number of depth cues provided by the visual environment. In recent years, many studies have been carried out to investigate the

The difference between the way in which binocular disparity scales with viewing distance and the way in which motion parallax scales with viewing distance introduces a potential indirect cue for viewing distance: the viewing distance is the only distance at which disparity and motion specify the same depth. The present study examines whether this information is used. Two simulated ellipsoids were presented on a computer screen in complete darkness. The two ellipsoids were 6 to the left and right of straight ahead. Subjects set the width and depth of each ellipsoid to match a tennis ball, and set the distance of the one on the right to half that of the one on the left. The distance of the left ellipsoid varied between trials. On half of the trials it was static. On the other half it was rotating up and down around its frontal horizontal axis. Rotating the left ellipsoid influenced its set depth: rotating ellipsoids were set to be much more spherical. There was no influence on the set depth of the other ellipsoid, or on the set width of either. The set distance of the right ellipsoid was also unaffected. We conclude that subjects do not combine binocular disparity and motion parallax to obtain more veridical information about viewing distance. 1999 Elsevier Science Ltd. All rights reserved.

The interaction of the depth cues of binocular disparity and motion parallax could potentially be used by the visual system to recover an estimate of the viewing distance. The present study investigated whether an interaction of stereo and motion has e#ects that persist over time to influence the perception of shape from stereo when the motion information is removed. Static stereoscopic ellipsoids were presented following the presentation of rotating stereoscopic ellipsoids, which were located either at the same or a di#erent viewing distance. It was predicted that shape judgements for static stimuli would be better after presentation of a rotating stimulus at the same viewing distance, than after presentation of one at a different viewing distance. No such di#erence was found. It was concluded that an interaction between stereo and motion depth cues does not influence the perception of subsequently presented static objects.

The ability to detect surfaces was studied in a multiple-cue condition in which binocular disparity and motion parallax could specify independent depth configurations. On trials on which binocular disparity and motion parallax were presented together, either binocular disparity or motion parallax could indicate a surface in one of two intervals; in the other interval, both sources indicated a volume of random points. Surface detection when the two sources of information were present and compatible was not better than detection in baseline conditions, in which only one source of information was present. When binocular disparity and motion specified incompatible depths, observers’ ability to detect a surface was severely impaired if motion indicated a surface but binocular disparity did not. Performance was not as severely degraded when binocular disparity indicated a surface and motion did not. This dominance of binocular disparity persisted in the presence of foreknowledge about which source of information would be relevant.

that “depth-maps-with-parameters ” are first computed in a modular fashion for each cue type. The resulting estimates are then promoted: the missing parameters in each depth map are filled in by comparison with others. The process of cue promotion constitutes an interaction among different cues, 2 FIGURE 1. Illustration of the manner in which depth from texture (d,) and depth from motion (d,,,) are varied independently. Round texture elements of random size and spacing are formed by the intersections of a cylinder (with depth d,) with randomly-placed balls. The resulting textured surface is then projected (parallel to z-axis) onto a second cylinder (with depth d,,,). Stimuli are generated by projections (parallel to z-axis) of the second surface rotated about the x-axis. The width of the projected surface is 2 W.