Recent studies of eye movements in reading and other information processing tasks, such as music reading, typing, visual search, and scene perception, are reviewed. The major emphasis of the review is on reading as a specific example of cognitive processing. Basic topics discussed with respect to reading are (a) the characteristics of eye movements, (b) the perceptual span, (c) integration of information across saccades, (d) eye movement control, and (e) individual differences (including dyslexia). Similar topics are discussed with respect to the other tasks examined. The basic theme of the review is that eye movement data reflect moment-to-moment cognitive processes in the various tasks examined. Theoretical and practical considerations concerning the use of eye movement data are also discussed. Many studies using eye movements to investigate cognitive processes have appeared over the past 20 years. In an earlier review, I (Rayner, 1978b) argued that since the mid-1970s we have been in a third era of eye movement research and that the success of research in the current era would depend on the ingenuity of researchers in designing interesting and informative

This paper examines the nature of visual representations that direct ongoing performance in sensorimotor tasks. Performance of such natural tasks requires relating visual information from different gaze positions. To explore this we used the technique of making task relevant display changes during saccadic eye movements. Subjects copied a pattern of colored blocks on a computer monitor, using the mouse to drag the blocks across the screen. Eye position was monitored using a dual-purkinje eye tracker, and the color of blocks in the pattern was changed at different points in task performance. When the target of the saccade changed color during the saccade, the duration of fixations on the model pattern increased, depending on the point in the task that the change was made. Thus different fixations on the same visual stimulus served a different purpose. The results also indicated that the visual information that is retained across successive fixations depends on moment by moment task dema...

This paper investigates the temporal dependencies of natural vision by measuring eye and hand movements while subjects made a sandwich. The phenomenon of change blindness suggests these temporal dependencies might be limited. Our observations are largely consistent with this, suggesting that much natural vision can be accomplished with “just-in-time ” representations. However, we also observe several aspects of performance that point to the need for some representation of the spatial structure of the scene that is built up over different fixations. Patterns of eye-hand coordination and fixation sequences suggest the need for planning and coordinating movements over a period of a few seconds. This planning must be in a coordinate frame that is independent of eye position, and thus requires a representation of the spatial structure in a scene that is built up over different fixations.

This paper presents the case for a functional account of vision. A variety of studies have consistently revealed “change blindness ” or insensitivity to changes in the visual scene during an eye movement. These studies indicate that only a small part of the information in the scene is represented in the brain from moment to moment. It is still unclear, however, exactlywhat is included in visual representations. This paper reviews experiments using an extended visuo-motor task, showing that display changes affect performance differently depending on the observer’s place in the task. These effects are revealed by increases in fixation duration following a change. Different task-dependent increases suggest that the visual system represents only the information that is necessary for the immediate visual task. This allows a principled exploration of the stimulus properties that are included in the internal visual representation. The task specificity also has a more general implication that vision should be conceptualized as an active process executing special purpose “routines ” that compute only the currently necessary information. Evidence for this view and its implications for visual representations are discussed. Comparison of the change blindness phenomenon and fixation durations shows that conscious report does not reveal the extent of the representations computed by the routines.

Below is the unedited, uncorrected final draft of a BBS target article that has been accepted for publication. This preprint has been prepared for potential commentators who wish to nominate themselves for formal commentary invitation. Please DO NOT write a commentary until you receive a formal invitation. If you are invited to submit a commentary, a copyedited, corrected version of this paper will be posted.

Recent physiological experiments have shown that the responses of many neurons in V1 and V3a are modulated by the direction of gaze. We have developed a neural network model of the hierarchy of maps in visual cortex to explore the hypothesis that visual features are encoded in egocentric (spatiotopic) coordinates at early stages of visual processing. Most psychophysical studies that have attempted to examine this question have concluded that features are represented in reti-nal coordinates, but the interpretation of these experiments does not preclude the type of retinospatiotopic representation that is embodied in our model. The model also explains why electrical stimulation experiments in visual cortex cannot distinguish between retinal and retinospatiotopic coordinates in the early stages of visual processing. Psychophysical predictions are made for testing the existence of retinospatiotopic representations. The three most common types of coordinates used for representing visual objects in biological and computer vision systems are eye-centered (retinal), object-centered, and viewer-centered (egocentric). Eye-centered representations are consistent with many physiological and psychophysical studies of early vision. Neurons in primary visual cortex and most extrastriate areas are organized into retinotopic maps (Felleman & Van Essen, 1991). Most psychophysical experiments designed to determine the nature of spatial representation at early visual stages have reached the same conclusion, as we review in the discussion. Other types of representation are believed to be used at the highest stages of visual processing. Mishkin, Ungerleider, and Macko (1983) proposed a functional distinction between two main streams of processing, the "what " and "where " pathways leading, respectively, in the temporal and parietal cortex. Object-centered reference frames have been suggested for the representation of objects in the inferior temporal cortex and egocentric reference frames have been proposed for the representation of spatial location in the parietal cortex (Andersen,

We recently demonstrated that the perceived stability of a visual target that is displaced during a saccade critically depends on whether the target is present immediately when the saccade ends; blanking a target during and just after a saccade makes its intra-saccadic displacement more visible (Deubel et al. Vis Res 1996;36:985–996). Here, we investigate the interaction of visual context and blanking. Subjects saw a saccade target and an equal-sized distractor. During a saccade one or the other was displaced left or right. At the same time, one of the objects could be blanked briefly. Subjects reported whether the target or the distractor had jumped. The object that was blanked was more often seen as jumping (Experiment 1), regardless of which object really jumped, implying that continuously visible objects are preferentially perceived as stable. When both objects were blanked, longer blanking led to better accuracy at identifying which had jumped during a saccade. When one object was jumped and the other, stationary object was blanked (Experiment 2), the blanked object was mistakenly seen as jumping until the jump covered 50 % or more of the saccade amplitude. In Experiment 3 a large continuously present texture underwent an undetected jump during a saccade, biasing judgments of simultaneous jumps of a blanked target. The results demonstrate that space constancy in normal situations is dominated by the assumption that a continuously present pattern is stable—this pattern becomes the spatial

An algorithm for the detection of visually relevant luminance features is presented. The algorithm is motivated and directed by current models of the visual system. The algorithm detects edges (sharp luminance transitions) and narrow bars (luminance cusps) and marks them with the proper polarity. The image is first bandpass filtered with oriented filters at a number of scales an octave apart. The suprathreshold image contrast details at each scale are then identified and are compared across scales to find locations in which the signal polarity (sign) is identical at all scales, representing a minimal level of phase congruence across scales. These locations maintain the polarity of the bandpass-filtered image. The result is a polarity-preserving features map representing the edges with pairs of light and dark lines or curves on corresponding sides of the contour. Similarly, bar features are detected and represented with single curves of the proper polarity. The algorithm is implemented without free (fitted) parameters. All parameters are directly derived from visual models and from measurements on human observers. The algorithm is shown to be robust with respect to variations in filter parameters and requires no use of quadrature filters or Hilbert transforms. The possible utility of such an algorithm within the visual system and in computer vision applications is discussed. Keywords—Biological systems, edge detection, image matching, image processing, machine vision. I.

The visual world appears unified, stable, and continuous despite rapid changes in eye position. How this is accomplished has puzzled psychologists for over a century. One possibility is that visual information from successive eye fixations is fused in memory according to environmental or spatiotopic coordinates. Evidence supporting this hypothesis was provided by Davidson, Fox, and Dick (1973). They presented a letter array in one fixation and a mask at one letter position in a subsequent fixation and found that the mask inhibited report of the letter that shared its retinal coordinates but appeared to occupy the same position as the letter that shared its spatial coordi-nates. This suggests the existence of a retinotopic visual persistence at which transsac-cadic masking occurs and a spatiotopic visual persistence at which transsaccadic integration, or fusion, occurs. Using a similar procedure, we found retinotopic masking and retinotopic integra-tion: The mask interfered with the letter that shared its retinal coordinates, but also appeared to cover that letter. In another experiment, instead of a mask we presented a bar marker over one letter position, and subjects reported the letter that appeared

Abstract: Why and how people perceive the visual world as continuous and stable, despite the gross changes of its retinal projection that occur with each saccade, is one of the classic problems in perception. In the present paper, we argue that an important factor of visual stability and transsaccadic perception is formed by the reafferent visual information, i.e., the visual display that is present when the eyes land. After a review of some of the relevant theoretical, behavioural and physiological research on space constancy, saccadic suppression and transsaccadic memory, three experiments are presented. In a first experiment, we study the effect of an extended horizontal bar covering the target area for a short period after the saccade on saccadic suppression of image displacement. The results show that the bar acts just like a temporary blanking of the saccade target, leading to a strong reduction of saccadic suppression. In the second experiment, we show that any object that is present immediately after the saccade can establish a spatial reference, even if it is dissimilar to the saccade target. In a third experiment we study, with a similar approach, the effect of blanking and postsaccadic information on transsaccadic integration of form information. The data demonstrate that a localized postsaccadic object tends to replace the content of transsaccadic memory.