How predictable are human eye movements during search in real world scenes? We recorded 14 observers ’ eye movements as they performed a search task (person detection) in 912 outdoor scenes. Observers were highly consistent in the regions fixated during search, even when the target was absent from the scene. These eye movements were used to evaluate computational models of search guidance from three sources: saliency, target features, and scene context. Each of these models independently outperformed a cross-image control in predicting human fixations. Models that combined sources of guidance ultimately predicted 94 % of human agreement, with the scene context component providing the most explanatory power. None of the models, however, could reach the precision and fidelity of an attentional map defined by human fixations. This work puts forth a benchmark for computational models of search in real world scenes. Further improvements in Please address all correspondence to Aude Oliva, Department of Brain and Cognitive

To make progess in understanding human visuo-motor behavior, we will need to understand its basic components at an abstract level. One way to achieve such an understanding would be to create a model of a human that has a sufficient amount of complexity so as to be capable of generating such behaviors. Recent technological advances have been made that allow progress to be made in this direction. Graphics models that simulate extensive human capabilities can be used as platforms from which to develop synthetic models of visuo-motor behavior. Currently such models can capture only a small portion of a full behavioral repertoire, but for the behaviors that they do model, they can describe complete visuo-motor subsystems at a useful level of detail. The value in doing so is that the body’s elaborate visuo-motor structures greatly simplify the specification of the abstract behaviors that guide them. The net result is that, essentially, one is behaviors at each instant. This paper outlines one such model. A centerpiece of the model uses vision to aid the behavior that has the most to gain from taking environmental measurements. Preliminary tests of the model against human performance in realistic VR environments show that main features of the model show up in human behavior. Categories and Subject Descriptors: I.2.10 [Vision and Scene Understanding]: Perceptual reasoning 1.

Attention and working memory limitations set strict limits on visual representations, yet researchers have little appreciation of how these limits constrain the acquisition of information in ongoing visually guided behavior. Subjects performed a brick sorting task in a virtual environment. A change was made to 1 of the features of the brick being held on about 10 % of trials. Rates of change detection for feature changes were generally low and depended on the pick-up and put-down relevance of the feature to the sorting task. Subjects ’ sorting decision suggests that changes may be missed because of a failure to update the changed feature. The authors also explore how hand and eye behavior are coordinated for strategic acquisition and storage of visual information throughout the task.

Prediction of gaze behavior in gaming environments can be a tremendously useful asset to game designers, enabling them to improve gameplay, selectively increase visual fidelity, and optimize the distribution of computing resources. The use of saliency maps is currently being advocated as the method of choice for predicting visual attention, crucially under the assumption that no specific task is present. This is achieved by analyzing images for low-level features such as motion, contrast, luminance, etc. However, the majority of computer games are designed to be easily understood and pose a task readily apparent to most players. Our psychophysical experiment shows that in a task-oriented context such as gaming, the predictive power of saliency maps at design time can be weak. Thus, we argue that a more involved protocol utilizing eye tracking, as part of the computer game design cycle, can be sufficiently robust to succeed in predicting fixation behavior of players.

You might find this additional information useful... This article cites 46 articles, 23 of which you can access free at:

mechanisms for gaze direction in interactive visual environments

642 M. M. Hayhoe et al. How do the limitations of attention and working memory constrain acquisition of information in the context of natural behavior? Overt fixations carry much information about current attentional state, and are a revealing indicator of this process. Fixation patterns in natural behavior are largely determined by the momentary task. The implication of this is that fixation patterns are a learnt behavior. We review several recent findings that reveal some aspects of this learning. In particular, subjects learn the structure and dynamic properties of the world in order to fixate critical regions at the right time. They also learn how to allocate attention and gaze to satisfy competing demands in an optimal fashion, and are sensitive to changes in those demands. Understanding exactly how tasks exert their control on gaze is a critical issue for future research.

First, and foremost, I would like to thank my family. My mother, father and brother have always supported me throughout my academic career (and my life in general). My mother is always full of words of encouragement and she has taught me that you can do anything you set your mind to. My father has been a pillar of support, providing useful advice at every moment. I really don’t know where I would be without his advice. He has helped me whenever I have asked no matter what the circumstances are and he has taught me that there is no substitute for hard-work. My brother has provided direction for me since I was born and has been an unbelievable role model. He certainly reminds me of the important things in life. I would also like to thank Krista Langkamer for not only believing in me, but for taking the time to actually help me accomplish what I wanted to accomplish. She always seems to make time to read over papers and make comments on talks despite the million things she needs to get done. None of this would have been possible without my close friends, Rumi Rahman and Dustin Scott, who have always been around when I needed to blow off steam; good friends are hard to find. I also want to thank Julisara Mathew for making graduate school classes “fun”.

The current paper proposes a novel model for integrative learning of proactive visual attention and sensory-motor control as inspired by the premotor theory of visual attention. The model is characterized by coupling a slow dynamics network with a fast dynamics network and by inheriting our prior proposed multiple timescales recurrent neural networks model (MTRNN) that may correspond to the fronto-parietal networks in the cortical brains. The neuro-robotics experiments in a task of manipulating multiple objects utilizing the proposed model demonstrated that some degrees of generalization in terms of position and object size variation can be achieved by organizing seamless integration of the proactive object-related visual attention and the related sensory-motor control into a set of action primitives in the distributed neural activities appearing in the fast dynamics network. It was also shown that such action primitives can be combined in compositional ways in acquiring novel actions in the slow dynamics network. The experimental results presented substantiate the premotor theory of visual attention.

Gaze analysis and prediction in interactive virtual environments, such as games, is a challenging topic since the 3D perspective and variations of the viewpoint as well as the current task introduce many variables that affect the distribution of gaze. In this article, we present a novel pipeline to study eye-tracking data acquired from interactive 3D applications. The result of the pipeline is an importance map which scores the amount of gaze spent on each object. This importance map is then used as a heuristic to predict a user’s visual attention according to the object properties present at runtime. The novelty of this approach is that the analysis is performed in object space and the importance map is defined in the feature space of high-level properties. High-level properties are used to encode task relevance and other attributes, such as eccentricity, which may have an impact on gaze behavior. The pipeline has been tested with an exemplary study on a first-person shooter game. In particular, a protocol is presented describing the data acquisition procedure, the learning of different importance maps from the data, and finally an evaluation of the performance of the derived gaze predictors. A metric measuring the degree of correlation between attention predicted by the importance map and the actual gaze yielded clearly positive results. The correlation becomes particularly strong when the player is attentive to an in-game task.