We examined how depth information from two different cue types (object motion and texture gradient) is integrated into a single estimate in human vision. Two critical assumptions of a recent model of depth cue combination (termed modified weak fusion) were tested. The first assumption is that the overall depth estimate is a weighted linear combination of the estimates derived from the individual cues, after initial processing needed to bring them to a common format. The second assumption is that the weight assigned to a cue reflects the apparent reliability of that cue in a particular scene. By this account, the depth combination rule is linear and dynamic, changing in a predictable fashion in response to the particular scene and viewing conditions. A novel procedure was used to measure the weights assigned to the texture and motion cues across experimental conditions. This procedure uses a type of perturbation analysis. The results are consistent with the weighted linear combination rule. In addition, when either cue is corrupted by added noise, the weighted linear combination rule shifts in favor of the uncontaminated cue.

This report describes research done within the Artificial Intelligence Laboratory and the Center for Biological Information Processing (Whitaker College) at the Massachusetts Institute of Technology. Support for the A.I. Laboratory 's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Department of Defense under Office of Naval Research contract N00014-85-K-0124. Support for this research is also provided by the Alfred P. Sloan Foundation, the Office of Naval Research, Cognitive and Neural Systems Division, the National Science Foundation and the McDonnell Foundation

The accuracy of depth judgments that are based on binocular disparity or structure from motion (motion parallax and object rotation) was studied in 3 experiments. In Experiment 1, depth judgments were recorded for computer simulations of cones specified by binocular disparity, motion parallax, or stereokinesis. In Experiment 2, judgments were recorded for real cones in a structured environment, with depth information from binocular disparity, motion parallax, or object rotation about the y-axis. In both of these experiments, judgments from binocular disparity information were quite accurate, but judgments on the basis of geometrically equivalent or more robust motion information reflected poor recovery of quantitative depth information. A 3rd experiment demonstrated stereoscopic depth constancy for distances of 1 to 3 m using real objects in a well-illuminated, structured viewing environment in which monocular depth cues (e.g., shading) were minimized. It has been pointed out that the geometric information supporting the perception of depth from binocular disparity is actually less determinate than that supporting the recovery of structure from object rotation or motion parallax

This paper addresses the computational role that the construction of a complete surface representation may play in the recovery of 3-D structure from motion. We first discuss the need to integrate surface reconstruction with the structure-from-motion process, both on computational and perceptual grounds. We then present a model that combines a feature-based structure-from-motion algorithm with a smooth surface interpolation mechanism. This model allows multiple surfaces to be represented in a given viewing direction, incorporates constraints on surface structure from object boundaries, and segregates image features onto multiple surfaces on the basis of their 2-D image motion. We present the results of computer simulations that relate the qualitative behavior of this model to psychophysical observations. In a companion paper, we discuss further perceptual observations regarding the possible role of surface reconstruction in the human recovery of 3-D structure from motion. Three-dimensional structure-from-motion perception Temporal integration Surface reconstruction Motion interpretation Motion

Observers viewed monocular animations of rotating dihedral angles and were required to indicate their perceived structures by adjusting the magnitude and orientation of a stereoscopic dihedral angle. The motion displays were created by directly manipulating various aspects of the image velocity field, including the mean translation, the horizontal and vertical velocity gradients, and the manner in which these gradients changed over time. The adjusted orientation of each planar facet was decomposed into components of slant and tilt. Although the tilt component was estimated with a high degree of accuracy, the judgments of slant exhibited large systematic errors. The magnitude of perceived slant was determined primarily by the magnitude of the velocity gradient scaled by its direction. The results also indicate that higher order temporal derivatives of the moving elements had little effect on observers’ judgments. A fundamental issue in the theoretical analysis of threedimensional (3-D) structure from motion concerns the number of distinct views that are required for different types of perceptual judgments. Whereas the first-order relations between pairs of views provide sufficient information

Perceived depth in the stereokinetic effect (SKE) illusion and in the monocular derivation of depth from motion parallax were compared. Motion parallax gradients of velocity can be decomposed into 2 components: object- and observer-relative transformations. SKE displays present only the object-relative component. Observers were asked to estimate the magnitude and near-far order of depth in motion parallax and SKE displays. Monocular derivation of depth magnitude from motion parallax is fully accounted for by the perceptual response to the SKE, and observerrelative transformations absent in the SKE are of perceptual utility only as determinants of the near-far signing of perceived sequential depth. The amount of depth and rigidity perceived in motion parallax and SKE displays covaries with the projective size of the stimuli. The monocular derivation of depth from motion is mediated by a perceptual heuristic of which the SKE is symptomatic. The perception of depth from monocular motion information occurs in three situations, two yielding veridical depth percepts and the third giving rise to an illusion. First, depth is perceived when one views an object that is rotating

Perceived orientation of axis of rotation and accuracy in discriminating fixed-axis from nordixed-axis rotations were investigated for orthographic projections of three-dimensional rotating objects. The principal findings were (a) the slant of the axis of rotation was systematically misperceived; (b) in both two-view and multiview displays, the perceived slant of the axis of rotation was well-predicted by the ratio between the deformation (a property of the first-order optic flow) and the component parallel to the image plane of the global velocity vector; (c) if this ratio was kept constant in each frame transition of the stimulus sequence (or it was varied), then the stimuli tended to be judged as fixed-axis rotations (or as nonfixed-axis rotations), regardless of whether they simulated a fixed-axis rotation or not; and (d) the tilt of the axis of rotation was perceived in two-view displays with a very small error. A changing two-dimensional (2-D) projection of an object's motion gives rise to a compelling impression of a volumetric shape moving in three-dimensional (3-D) space. This phenomenon (called the kinetic depth effect [KDE] after Wallach & O'Cormell, 1953) represents an essential

In four experiments, a scalar judgment of perceived depth was used to examinethe spatial and temporal characteristics of the perceptual buildup of three-dimensional (3-D) structure from optical motion as a function of the depth in the simulated object, the speed of motion, the number of elements defining the object, the smoothness of the optic flow field, and the type of motion. In most of the experiments, the objects were polar projections of simulatedhalf-ellipsoids undergoing a curvilinear translation about the screen center. It was foundthat-thebuiklupof3-D structure was: (1) jointly dependent on the speed at which an object moved and on the range through which the object moved; (2) more rapid for deep simulated objects than for shallow objects; (3) unaffected by the number of points defining the object, including the maximum apparent depth within each simulated object-depth condition; (4) not disrupted by nonsmooth optic flow fields; and (5) more rapid for rotating objects than for curvilinearly translating objects. The human visual system has the remarkable ability to recover three-dimensional (3-D) shape when it is presented with a rapid succession of 2-D views of a moving object. Even when each view by itself contains no information about three-dimensionality, depth can still be perceived. In their now classic study, Wallach and O’Connell (1953) named this phenomenon the kinetic depth effect (KDE). Recent investigators have called the phenomenon the recovery of structurefrom motion (SFM) (e.g., Todd, 1984; Ullman, 1979, 1984). When viewing a KDE display, one often has the impression that the time course for the structural buildup is quite short. Wallach and O’Connell (1953) took note of this factwhen they observed that “turning wire-figures were seen threedimensionally immediately upon presentation” (p. 208). Surprisingly, until recent years, there was very little data about the temporal characteristics of the process involved in the recovery of SFM (see Hildreth,

This study presents three ¢ndings concerning the mechanisms of depth perception. First, the shape of the three-dimensional percept evoked by two-frame motion is de¢ned solely by the rotation component around an axis in the frontoparallel plane; the visual system assigns a default value to this rotation component to arrive at a unique solution. Second, when the visual axes of two eyes are almost parallel, the visual system uses a default vergence value to reconstruct stereoscopic depth. Third, the default vergence and default rotation angles are highly correlated across subjects. This correlation implies that the two modalities share a common scaling default at an internal level.

that “depth-maps-with-parameters ” are first computed in a modular fashion for each cue type. The resulting estimates are then promoted: the missing parameters in each depth map are filled in by comparison with others. The process of cue promotion constitutes an interaction among different cues, 2 FIGURE 1. Illustration of the manner in which depth from texture (d,) and depth from motion (d,,,) are varied independently. Round texture elements of random size and spacing are formed by the intersections of a cylinder (with depth d,) with randomly-placed balls. The resulting textured surface is then projected (parallel to z-axis) onto a second cylinder (with depth d,,,). Stimuli are generated by projections (parallel to z-axis) of the second surface rotated about the x-axis. The width of the projected surface is 2 W.