Various visual cues provide information about depth and shape in a scene. When several of these cues are simultaneously available in a single location in the scene, the visual system attempts to combine them. In this paper, we discuss three key issues relevant to the experimental analysis of depth cue combination in human vision: cue promotion, dynamic weighting of cues, and robustness of cue combination. We review recent psychophysical studies of human depth cue combination in light of these issues. We organize the discussion and review as the development of a model of the depth cue combination process termed modified weak fusion (MWF). We relate the MWF framework to Bayesian theories of cue combination. We argue that the MWF model is consistent with previous experimental results and is a parsimonious summary of these results. While the MWF model is motivated by normative considerations, it is primarily intended to guide experimental analysis of depth cue combination in human vision. We describe experimental methods, analogous to perturbation analysis, that permit us to analyze depth cue combination in novel ways. In particular these methods allow us to investigate the key issues we have raised. We summarize recent experimental tests of the MWF framework that use these methods. Depth Multiple cues Sensor fusion

Abstract not found

We examined how depth information from two different cue types (object motion and texture gradient) is integrated into a single estimate in human vision. Two critical assumptions of a recent model of depth cue combination (termed modified weak fusion) were tested. The first assumption is that the overall depth estimate is a weighted linear combination of the estimates derived from the individual cues, after initial processing needed to bring them to a common format. The second assumption is that the weight assigned to a cue reflects the apparent reliability of that cue in a particular scene. By this account, the depth combination rule is linear and dynamic, changing in a predictable fashion in response to the particular scene and viewing conditions. A novel procedure was used to measure the weights assigned to the texture and motion cues across experimental conditions. This procedure uses a type of perturbation analysis. The results are consistent with the weighted linear combination rule. In addition, when either cue is corrupted by added noise, the weighted linear combination rule shifts in favor of the uncontaminated cue.

A global shape judgement task was used to investigate the combination of stereopsis and kinetic depth. With botb cues present, there were no distortions of shape perception, even under conditions where either cue alone did show such distortions. We suggest that the addition of motion information overcomes the stereo distance scaling problem. However, when incongruent combinations of disparity and motion were used, the results did not match predictions of a number of combination theories. These data could be described by a model which used weighted linear combination afier correctly scaling disparities for viewing distance. When the motion cue was weakened by presenting only two frames of each motion sequence, stereo was weighted more heavily. Stereopsis Structure-from-motion Three-dimensional shape perception Integration of depth cues

The accuracy of depth judgments that are based on binocular disparity or structure from motion (motion parallax and object rotation) was studied in 3 experiments. In Experiment 1, depth judgments were recorded for computer simulations of cones specified by binocular disparity, motion parallax, or stereokinesis. In Experiment 2, judgments were recorded for real cones in a structured environment, with depth information from binocular disparity, motion parallax, or object rotation about the y-axis. In both of these experiments, judgments from binocular disparity information were quite accurate, but judgments on the basis of geometrically equivalent or more robust motion information reflected poor recovery of quantitative depth information. A 3rd experiment demonstrated stereoscopic depth constancy for distances of 1 to 3 m using real objects in a well-illuminated, structured viewing environment in which monocular depth cues (e.g., shading) were minimized. It has been pointed out that the geometric information supporting the perception of depth from binocular disparity is actually less determinate than that supporting the recovery of structure from object rotation or motion parallax

Abstract not found

How the eye measures reality and virtual reality We, as a species, seem to have been fascinated with pictures throughout our history. The paintings at Niaux, Altamira, and Lascaux (Clottes, 1995; Ruspoli, 1986), for example, are known to be about 14,000 years old, but with the recently discovered paintings in the Grotte Chauvet, the origin of representational art appears to have been pushed back even further (Chauvet, Brunel Deschamps, & Hillaire, 1995; Clottes, 1996), to 20,000 years ago if not longer. 1 Thus, these paintings date from about the time at which homo sapiens sapiens first appeared in Europe (Nougier, 1969). We should remember these paintings in the context of virtual reality; our fascination with pictures is by no means recent. My intent is threefold: first, to discuss our perception of the cluttered layout, or space, that we normally find around us; second, to discuss the development of representational art up to our current appreciation of it; and third, to apply this knowledge to virtual reality systems. The first discussion focuses on the use of multiple sources of information specifying ordinal depth relations, within the theoretical framework that I have called directed perception

Abstract not found

that “depth-maps-with-parameters ” are first computed in a modular fashion for each cue type. The resulting estimates are then promoted: the missing parameters in each depth map are filled in by comparison with others. The process of cue promotion constitutes an interaction among different cues, 2 FIGURE 1. Illustration of the manner in which depth from texture (d,) and depth from motion (d,,,) are varied independently. Round texture elements of random size and spacing are formed by the intersections of a cylinder (with depth d,) with randomly-placed balls. The resulting textured surface is then projected (parallel to z-axis) onto a second cylinder (with depth d,,,). Stimuli are generated by projections (parallel to z-axis) of the second surface rotated about the x-axis. The width of the projected surface is 2 W.

The three-dimensional space around us is conveniently and reasonably Euclidean. Can we assume our perception of this space, and of objects in it, would follow suit? This assumption is quite natural, but empirical results suggest that it is also quite wrong, except in narrow circumstances. Perceptual space grades from being nearly Euclidean within a meter of our eyes to being affine and foreshortened at increasing distance, although considerable variation occurs across task, environments, and individuals. Compression with distance is steeper than one modeled by an exponent. Since pictures are most typically composed with distant content, similar perceived distortions should and do occur in pictorial, particularly photographic, space. One can think of perceived space—even at is articulated, near-Euclidean best—as built up incrementally from constraints of ordinality. These constraints, when sufficiently rich, converge on a near-Euclidean framework. How do we perceive the space in pictures? In answering this question theorists typically consider standard photographs and other representational images, such as architectural drawings, engravings, and paintings in linear perspective. Adding motion augments this