Our knowledge of the way that the visual system operates in everyday behavior has, until recently, been very limited. This information is critical not only for understanding visual function, but also for understanding the consequences of various kinds of visual impairment, and for the development of interfaces between human and artificial systems. The development of eye trackers that can be mounted on the head now allows monitoring of gaze without restricting the observer's movements. Observations of natural behavior have demonstrated the highly task-specific and directed nature of fixation patterns, and reveal considerable regularity between observers. Eye, head, and hand coordination also reveals much greater flexibility and task-specificity than previously supposed. Experimental examination of the issues raised by observations of natural behavior requires the development of complex virtual environments that can be manipulated by the experimenter at critical points during task performance. Experiments where we monitored gaze in a simulated driving environment demonstrate that visibility of task relevant information depends critically on active search initiated by the observer according to an internally generated schedule, and this schedule depends on learnt regularities in the environment. In another virtual environment where observers copied toy models we showed that regularities in the spatial structure are used by observers to control eye movement targeting. Other experiments in a virtual environment with haptic feedback show that even simple visual properties like size are not continuously available or processed automatically by the visual system, but are dynamically acquired and discarded according to the momentary task demands.

The change blindness phenomenon suggests that visual representations retained across saccades are very limited. In this paper we sought to specify the kind of information that is in fact retained. We investigated targeting performance for saccadic eye movements, since one need for visual representations across eye and body positions may be to guide coordinated movements. We examined saccades in the context of an ongoing sensory motor task in order to make stronger generalizations about natural visual functioning and deployment of attention. Human subjects copied random patterns of coloured blocks on a computer display. Their eye movement pattern was consistent from block to block, including a precise saccade to a previously-placed, neighbouring block during each additional block placement.This natural, consistent eye movement allowed the previously-placed, neighbouring block to serve as an implicit target without instructions to the subject. On random trials, we removed the target object from the display during a preceding saccade, so that observers were required to make the targeting saccade without a currently visible target. Targeting performance was excellent, and appeared to be influenced by spatial information that was not visible during the preceding fixation. Subjects were generally unaware of the disappearance and reappearance of the target. We conclude that spatial information about visual targets is retained across eye movements and used to guide subsequent movements.

This paper reviews research examining the role of visual memory in scene perception and visual search. Recent theories in these literatures have held that coherent object representations in visual memory are fleeting, disintegrating upon the withdrawal of attention from an object. I discuss evidence demonstrating that, far from being transient, visual memory supports the accumulation of information from scores of individual objects in scenes, utilizing both visual short-term memory and visual long-term memory. In addition, I review evidence that memory for the spatial layout of a scene and memory for specific object positions can efficiently guide search within natural scenes. In the past decade, the interaction between perception and memory has received a great deal of attention from cognitive scientists. Much of this interest has originated from increased understanding that perception is a dynamic, serial process, extended over space and time. In this paper, I will discuss two related lines of research in which the relationship between perception and memory has come to the fore: Scene perception and visual

Visual short-term memory (VSTM) has received intensive study over the past decade, with research focused on VSTM capacity and representational format. Yet, the function of VSTM in human cognition is not well understood. Here, the authors demonstrate that VSTM plays an important role in the control of saccadic eye movements. Intelligent human behavior depends on directing the eyes to goal-relevant objects in the world, yet saccades are very often inaccurate and require correction. The authors hypothesized that VSTM is used to remember the features of the current saccade target so that it can be rapidly reacquired after an errant saccade, a task faced by the visual system thousands of times each day. In 4 experiments, memory-based gaze correction was accurate, fast, automatic, and largely unconscious. In addition, a concurrent VSTM load interfered with memory-based gaze correction, but a verbal short-term memory load did not. These findings demonstrate that VSTM plays a direct role in a fundamentally important aspect of visually guided behavior, and they suggest the existence of previously unknown links between VSTM representations and the occulomotor system.

integration of information [10]. Furthermore, points flashed around the time of a saccade are systematically mislocalized in a way that suggests slow build-up of compensation for retinal shifts [11--17]. The dynamic properties of neurons that remap their receptive fields in the anticipation of saccades may be closely connected with these distortions and contribute to spatial constancy [6]. Such neurons have been found in posterior parietal cortex [18--20], in superior colliculus [21], and in the frontal eye field [22] of monkeys; recently, neuroimaging has demonstrated similar spatial updating in human parietal cortex [23]. An aspect of the spatial constancy problem that has received little or no attention is the threedimensional stability of the world during eye movements. Vision serves not only for detecting the directions of points, but also---and at least as importantly---for extracting the three-dimensional layout of the environment, and in particular the orientations of surface

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : xiii I Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 A. Spatial representations and sensori-motor coordination : : : : : : : : : 1 B. The posterior parietal cortex : : : : : : : : : : : : : : : : : : : : : : : 2 C. Neural code for spatial representations : : : : : : : : : : : : : : : : : : 4 1. Dynamic remapping : : : : : : : : : : : : : : : : : : : : : : : : : : 4 2. Gain modulation : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6 3. The Zipser and Andersen Network : : : : : : : : : : : : : : : : : : 6 D. Parallel vectorial representations : : : : : : : : : : : : : : : : : : : : : 9 E. Thesis Outline : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10 1. Hierarchy in spatial representations : : : : : : : : : : : : : : : : : 10 2. A basis function approach for spatial representation : : : : : : : : 11 II Egocentric spatial representation in early vision : :...

It is generally accepted that there is something special about reasoning that uses mental images. The question of how it is special, however, has never been satisfactorily spelled out, despite over thirty years of research in the post-behaviorist tradition. This article considers some of the general motivation for the assumption that entertaining mental images involves inspecting a picture-like object. It sets out a distinction between phenomena attributable to the nature of mind, to what is called the cognitive architecture, and ones that are attributable to tacit knowledge used to simulate what would happen in a visual situation. With this distinction in mind the paper then considers in detail the widely held assumption that in some important sense images are spatially displayed or are depictive, and that examining images uses the same mechanisms that are deployed in visual perception. I argue that the assumption of the spatial or depictive nature of images is only explanatory if tak...

Contents 1. The puzzle of seeing......................................1-2 1.1 Why do things look the way they do? ..................................................................................1-2 1.2 What is seeing? .................................................................................................................1-3 1.3 Does vision create a "picture" in the head?...........................................................................1-4 1.3.1 The richness of visual appearances and the poverty of visual information: Reconciling the difference ...........................................................................................1-4 1.3.2 Some reasons for thinking there may be an inner display................................................1-6 1.4 Some problems with the Internal Display Assumption: Part I: What's in the display and how does it get there?....................................................................1-10 1.4.1 How is

Facing the competing demands for wider field of view and higher spatial resolution, computer vision will evolve toward greater use of foveal sensors and frequent camera movements. Integra- tion of visual information across movements be- comes a fundamental problem. We show that integration is possible using a biologically-inspired representation we call the visual-motor calibration map. The map is a memory-based model of the relationship between camera movements and corresponding pixel locations before and after any movement. The map constitutes a selfcalibration that can compensate for non-uniform sampling, lens distortion, mechanical misalignments, and arbitrary pixel reordering. Integration takes place entirely in a retinotopic frame, using a short-term, predictive visual memory.

Abstract not found