A visual attention system, inspired by the behavior and the neuronal architecture of the early primate visual system, is presented. Multiscale image features are combined into a single topographical saliency map. A dynamical neural network then selects attended locations in order of decreasing saliency. The system breaks down the complex problem of scene understanding by rapidly selecting, in a computationally efficient manner, conspicuous locations to be analyzed in detail. Index terms: Visual attention, scene analysis, feature extraction, target detection, visual search. \Pi I. Introduction Primates have a remarkable ability to interpret complex scenes in real time, despite the limited speed of the neuronal hardware available for such tasks. Intermediate and higher visual processes appear to select a subset of the available sensory information before further processing [1], most likely to reduce the complexity of scene analysis [2]. This selection appears to be implemented in the ...

The classical model of visual processing in cortex is a hierarchy of increasingly sophisticated representations, extending in a natural way the model of simple to complex cells of Hubel and Wiesel. Somewhat surprisingly, little quantitative modeling has been done in the last 15 years to explore the biological feasibility of this class of models to explain higher level visual processing, such as object recognition. We describe a new hierarchical model that accounts well for this complex visual task, is consistent with several recent physiological experiments in inferotemporal cortex and makes testable predictions. The model is based on a novel MAX-like operation on the inputs to certain cortical neurons which may have a general role in cortical function.

Abstract not found

It is well established that attention modulates visual processing in extrastriate cortex. However, the underlying neural mechanisms are unknown. A consistent observation is that attention has its greatest impact on neuronal responses when multiple stimuli appear together within a cell’s receptive field. One way to explain this is to assume that multiple stimuli activate competing populations of neurons and that attention biases this competition in favor of the attended stimulus. In the absence of competing stimuli, there is no competition to be resolved. Accordingly, attention has a more limited effect on the neuronal response to a single stimulus. To test this interpretation, we measured the responses of neurons in macaque areas V2 and V4 using a behavioral paradigm that allowed us to isolate automatic sensory processing mechanisms from attentional effects. First, we measured each cell’s response to a single

this article.

When we look at images, certain salient structures often attract our immediate attention, without requiring a systematic scan of the entire image. In subsequent stages, processing resources can be allocated preferentially to these salient structures. In many cases this saJiency is a property of the structure as a whole, i.e., parts of the structure are not salient in isolation. In this paper we present a saliency measure based on cur- vature and curvature variation. The structures this measure emphasizes are also salient in human perception, and they often correspond to objects of interest in the image. We present a method for computing the sallehey by a simple iterative scheme, using a uniform network of locally connected processing elements. The network uses an optimization approach to produce a "saliency map" which is a representation of the image emphasizing salient locations. The main.properties of the network are: (i) the computations are simple and local, (ii) globally salient structures emerge with a small number of iterations (iii) as a by-product of the computation contours are smoothed, and gaps are filled-in.

Bottom-up or saliency-based visual attention allows primates to detect non-specific conspicuous targets in cluttered scenes. A classical metaphor, derived from electrophysiological and psychophysical studies, describes attention as a rapidly shiftable "spotlight". The model described here reproduces the attentional scanpaths of this spotlight: Simple multi-scale "feature maps" detect local spatial discontinuities in intensity, color, orientation or optical flow, and are combined into a unique "master" or "saliency" map. The saliency map is sequentially scanned, in order of decreasing saliency, by the focus of attention. We study the problem of combining feature maps, from different visual modalities and with unrelated dynamic ranges (such as color and motion), into a unique saliency map. Four combination strategies are compared using three databases of natural color images: (1) Simple normalized summation, (2) linear combination with learned weights, (3) global non-linear normalization...

The hypothesis that abrupt visual onsets capture attention automatically, as suggested by Yantis and Jonides (1984) was tested in four experiments. A centrally located cue directed attention to one of several stimulus positions in preparation for the identification of a target letter embedded in an army of distractor letters. In all experiments, one stimulus (either the target or one of the distractors) had an abrupt onset; the remaining letters did not. The effectiveness of the cue was manipulated (varying either its duration or its predictive validity) to test whether abrupt onsets capture attention even when subjects are in a highly focused attentional state. Results showed that onsets do not necessarily capture attention in violation of an observer's intentions. A mechanism for partially automatic attentional capture by abrupt onset is proposed, and the diagnosticity of the intentionality criterion for automaticity is discussed. Introspective and empirical evidence both suggest that the abrupt appearance of an object in the visual field "draws attention. " A plausible account of this phenomenon is that there exists a mechanism that is tuned to abrupt onsets and that one of its functions is to direct visual attention to the

INTRODUCTION Global illumination is the physically accurate calculation of lighting in an environment. It is computationally expensive for static environments and even more so for dynamic environments. Not only are many images required for an animation, but the calculation involved increases with the presence of moving objects. In static environments, global illumination algorithms can precompute a lighting solution and reuse it whenever the viewpoint changes, but in dynamic environments, any moving object or light potentially affects the illumination of every other object in a scene. To guarantee accuracy, the algorithm has to recompute the entire lighting solution for each frame. This paper describes a perceptually-based technique that can dramatically reduce this computational load. The technique may also be used in image based rendering, geometry level of detail selection, realistic image synthesis, video telephony and video compression. Perceptually-based rendering operat

Primates demonstrate unparalleled ability at rapidly orienting towards important events in complex dynamic environments. During rapid guidance of attention and gaze towards potential objects of interest or threats, often there is no time for detailed visual analysis. Thus, heuristic computations are necessary to locate the most interesting events in quasi real-time. We present a new theory of sensory surprise, which provides a principled and computable shortcut to important information. We develop a model that computes instantaneous low-level surprise at every location in video streams. The algorithm significantly correlates with eye movements of two humans watching complex video clips, including television programs (17,936 frames, 2,152 saccadic gaze shifts). The system allows more sophisticated and time-consuming image analysis to be efficiently focused onto the most surprising subsets of the incoming data. 1.