We show a learning-based method for low-level vision problems. We set-up a Markov network of patches of the image and the underlying scene. A factorization approximation allows us to easily learn the parameters of the Markov network from synthetic examples of image/scene pairs, and to e ciently propagate image information. Monte Carlo simulations justify this approximation. We apply this to the \super-resolution " problem (estimating high frequency details from a low-resolution image), showing good results. For the motion estimation problem, we show resolution of the aperture problem and lling-in arising from application of the same probabilistic machinery.

We seek the scene interpretation that best explains image data.

We argue that Bayesian decision theory provides a good theoretical framework for visual perception. Such a theory involves a likelihood function specifying how the scene generates the image(s), a prior assumption about the scene, and a decision rule to determine the scene interpretation. This is illustrated by describing Bayesian theories for individual visual cues and showing that perceptual biases found in psychophysical experiments can be interpreted as biases towards prior assumptions made by the visual system. We then describe the implications of this framework for the integration of different cues. We argue that the dependence of cues on prior assumptions means that care must be taken to model these dependencies during integration. This suggests that a number of proposed schemes for cue integration, which only allow weak interaction between cues, are not adequate and instead stronger coupling is often required. These theories require the choice of decision rules and we argue that...

A key problem in recognition is that the image of an object depends on the lighting conditions. We investigated whether recognition is sensitive to illumination using 3-D objects that were lit from either the left or right, varying both the shading and the cast shadows. In experiments 1 and 2 participants judged whether two sequentially presented objects were the same regardless of illumination. Experiment 1 used six objects that were easily discriminated and that were rendered with cast shadows. While no cost was found in sensitivity, there was a response time cost over a change in lighting direction. Experiment 2 included six additional objects that were similar to the original six objects making recognition more difficult. The objects were rendered with cast shadows, no shadows, and as a control, white shadows. With normal shadows a change in lighting direction produced costs in both sensitivity and response times. With white shadows there was a much larger cost in sensitivity and a...

The function of vision is to get correct and useful answers about the state of the world. However, given that the state of the world is not uniquely specified by the visual input, the visual system must make good guesses or inferences. Thus, theories of visual system functions will be theories of inference, and we need a language in which theories of inference can be described. Analogous to calculus having a minimum expressiveness required to formulate theories in physics, we argue that the language of Bayesian inference is fundamental to quantitatively describe how reliable answers about the world can be obtained from image patterns. Bayes provides a minimal formalism that can deal with the sophistication and versatility of perception missing from some other approaches. Key missing components include the ability to model uncertainty, probabilistic modeling of pattern synthesis as a necessary prerequisite to understanding pattern inference, the means to handle the complexity of natural images, and the diversity of visual tasks. Most of the formal elements that we describe are not new and have their roots in signal detection theory and ideal observer analysis. We start from there to review and codify principles drawn from recent applications of Bayesian decision theory, Bayes nets and pattern theory to vision. To emphasize the

The Bayesian approach to vision provides a fruitful theoretical framework both for modeling individual cues, such as stereo, shading, texture and occlusion, and for integrating their information. In this formalism we represent the viewed scene by one, or more, surfaces using prior assumptions about the surface shapes and material properties. On theoretical grounds, the less information available to the cues (and the less accurate it is) then the more important these assumptions become. This suggests that visual illusions, and biased perceptions, will arise for scenes for which the prior assumptions are not appropriate. We describe psychophysical experiments which are consistent with these ideas. Our Bayesian approach also has two important implications for coupling different visual cues. Firstly, different cues cannot in general be treated independently and then simply combined together at the end. There are dependencies between them that have to be incorporated into the models. Second...

Abstract. We describe a learning-based method for low-level vision problems—estimating scenes from images. We generate a synthetic world of scenes and their corresponding rendered images, modeling their relationships with a Markov network. Bayesian belief propagation allows us to efficiently find a local maximum of the posterior probability for the scene, given an image. We call this approach VISTA—Vision by Image/Scene TrAining. We apply VISTA to the “super-resolution ” problem (estimating high frequency details from a low-resolution image), showing good results. To illustrate the potential breadth of the technique, we also apply it in two other problem domains, both simplified. We learn to distinguish shading from reflectance variations in a single image under particular lighting conditions. For the motion estimation problem in a “blobs world”, we show figure/ground discrimination, solution of the aperture problem, and filling-in arising from application of the same probabilistic machinery.

Lightness illusions are fundamental to human perception, and yet why we see them is still the focus of much research. Here we address the question by modelling not human physiology or perception directly as is typically the case but our natural visual world and the need for robust behaviour. Artificial neural networks were trained to predict the reflectance of surfaces in a synthetic ecology consisting of 3-D ‘‘dead-leaves’ ’ scenes under non-uniform illumination. The networks learned to solve this task accurately and robustly given only ambiguous sense data. In addition—and as a direct consequence of their experience—the networks also made systematic ‘‘errors’ ’ in their behaviour commensurate with human illusions, which includes brightness contrast and assimilation—although assimilation (specifically White’s illusion) only emerged when the virtual ecology included 3-D, as opposed to 2-D scenes. Subtle variations in these illusions, also found in human perception, were observed, such as the asymmetry of brightness contrast. These data suggest that ‘‘illusions’ ’ arise in humans because (i) natural stimuli are ambiguous, and (ii) this ambiguity is resolved empirically by encoding the statistical relationship between images and scenes in past visual experience. Since resolving stimulus ambiguity is a challenge faced by all visual systems, a corollary of these findings is that human illusions must be experienced by all visual animals regardless of their particular neural machinery. The data also provide a more formal definition of illusion: the condition in which the true source of a stimulus differs from what is its most likely (and thus perceived) source. As such, illusions are not fundamentally different from non-illusory percepts, all being direct manifestations of the statistical relationship between images and scenes. Citation: Corney D, Lotto RB (2007) What are lightness illusions and why do we see them? PLoS Comput Biol 3(9): e180. doi:10.1371/journal.pcbi.0030180