Various visual cues provide information about depth and shape in a scene. When several of these cues are simultaneously available in a single location in the scene, the visual system attempts to combine them. In this paper, we discuss three key issues relevant to the experimental analysis of depth cue combination in human vision: cue promotion, dynamic weighting of cues, and robustness of cue combination. We review recent psychophysical studies of human depth cue combination in light of these issues. We organize the discussion and review as the development of a model of the depth cue combination process termed modified weak fusion (MWF). We relate the MWF framework to Bayesian theories of cue combination. We argue that the MWF model is consistent with previous experimental results and is a parsimonious summary of these results. While the MWF model is motivated by normative considerations, it is primarily intended to guide experimental analysis of depth cue combination in human vision. We describe experimental methods, analogous to perturbation analysis, that permit us to analyze depth cue combination in novel ways. In particular these methods allow us to investigate the key issues we have raised. We summarize recent experimental tests of the MWF framework that use these methods. Depth Multiple cues Sensor fusion

Abstract not found

We examined how depth information from two different cue types (object motion and texture gradient) is integrated into a single estimate in human vision. Two critical assumptions of a recent model of depth cue combination (termed modified weak fusion) were tested. The first assumption is that the overall depth estimate is a weighted linear combination of the estimates derived from the individual cues, after initial processing needed to bring them to a common format. The second assumption is that the weight assigned to a cue reflects the apparent reliability of that cue in a particular scene. By this account, the depth combination rule is linear and dynamic, changing in a predictable fashion in response to the particular scene and viewing conditions. A novel procedure was used to measure the weights assigned to the texture and motion cues across experimental conditions. This procedure uses a type of perturbation analysis. The results are consistent with the weighted linear combination rule. In addition, when either cue is corrupted by added noise, the weighted linear combination rule shifts in favor of the uncontaminated cue.

Previous investigators have shown that observers' visual cue combination strategies are remarkably flexible in the sense that these strategies adapt on the basis of the estimated reliabilities of the visual cues. However, these researchers have not addressed how observers' acquire these estimated reliabilities. This article studies observers' abilities to learn cue combination strategies. Subjects made depth judgments about simulated cylinders whose shapes were indicated by motion and texture cues. Because the two cues could indicate different shapes, it was possible to design tasks in which one cue provided useful information for making depth judgments, whereas the other cue was irrelevant. The results of experiment 1 suggest that observers' cue combination strategies are adaptable as a function of training; subjects adjusted their cue combination rules to use a cue more heavily when the cue was informative on a task versus when the cue was irrelevant. Experiment 2 demonstrated that experience-dependent adaptation of cue combination rules is context-sensitive. On trials with presentations of short cylinders, one cue was informative, whereas on trials with presentations of tall cylinders, the other cue was informative. The results suggest that observers can learn multiple cue combination rules, and can learn to apply each rule in the appropriate context. Experiment 3 demonstrated a possible limitation on the context-sensitivity of adaptation of cue combination rules. One cue was informative on trials with presentations of cylinders at a left oblique orientation, whereas the other cue was informative on trials with presentations of cylinders at a right oblique orientation. The results indicate that observers did not learn to use different cue combination rules in differe...

We argue that Bayesian decision theory provides a good theoretical framework for visual perception. Such a theory involves a likelihood function specifying how the scene generates the image(s), a prior assumption about the scene, and a decision rule to determine the scene interpretation. This is illustrated by describing Bayesian theories for individual visual cues and showing that perceptual biases found in psychophysical experiments can be interpreted as biases towards prior assumptions made by the visual system. We then describe the implications of this framework for the integration of different cues. We argue that the dependence of cues on prior assumptions means that care must be taken to model these dependencies during integration. This suggests that a number of proposed schemes for cue integration, which only allow weak interaction between cues, are not adequate and instead stronger coupling is often required. These theories require the choice of decision rules and we argue that...

Abstract not found

A global shape judgement task was used to investigate the combination of stereopsis and kinetic depth. With botb cues present, there were no distortions of shape perception, even under conditions where either cue alone did show such distortions. We suggest that the addition of motion information overcomes the stereo distance scaling problem. However, when incongruent combinations of disparity and motion were used, the results did not match predictions of a number of combination theories. These data could be described by a model which used weighted linear combination afier correctly scaling disparities for viewing distance. When the motion cue was weakened by presenting only two frames of each motion sequence, stereo was weighted more heavily. Stereopsis Structure-from-motion Three-dimensional shape perception Integration of depth cues

The notion of features of recognition, of varying complexities and possessing different degrees of spatial localization, offers a unified approach to the understanding of object recognition in human vision. According to this approach, the visual system synthesizes a feature-based recognition module for a given object when it is first encountered (cf.

Abstract not found

The Bayesian approach to vision provides a fruitful theoretical framework both for modeling individual cues, such as stereo, shading, texture and occlusion, and for integrating their information. In this formalism we represent the viewed scene by one, or more, surfaces using prior assumptions about the surface shapes and material properties. On theoretical grounds, the less information available to the cues (and the less accurate it is) then the more important these assumptions become. This suggests that visual illusions, and biased perceptions, will arise for scenes for which the prior assumptions are not appropriate. We describe psychophysical experiments which are consistent with these ideas. Our Bayesian approach also has two important implications for coupling different visual cues. Firstly, different cues cannot in general be treated independently and then simply combined together at the end. There are dependencies between them that have to be incorporated into the models. Second...