Abstract

We describe a neurobiological model of visual attention and eye/head movements in primates, and its application to the automatic animation of a realistic virtual human head watching an unconstrained variety of visual inputs. The bottom-up (image-based) attention model is based on the known neurophysiology of visual processing along the occipito-parietal pathway of the primate brain, while the eye/head movement model is derived from recordings in freely behaving Rhesus monkeys. The system is successful at autonomously saccading towards and tracking salient targets in a variety of video clips, including synthetic stimuli, real outdoors scenes and gaming console outputs. The resulting virtual human eye/head animation yields realistic rendering of the simulation results, both suggesting applicability of this approach to avatar animation and reinforcing the plausibility of the neural model. 1.

Citations

849	The Laplacian pyramid as a compact image code - Burt, Adelson - 1983
688	A model of saliency-based visual attention for rapid scene analysis - Itti, Koch, et al. - 1998
587	Receptive fields, binocular interaction and functional architecture in the cat's visual cortex - Hubel, N - 1962
461	A feature-integration theory of attention - Treisman, Gelade - 1980
396	Shifts in selective visual attention: Towards the under-laying neural circuitry, Human Neurobiology 4 - Koch, Ullman
256	Distance transformations in digital images - Borgefors - 1986
241	C (2000) A saliency-based search mechanism for overt and covert shifts of visual attention - Itti, Koch
220	Selective attention gates visual processing in the extrastriate cortex - Moran, Desimone - 1985
200	Modeling visual attention via selective tuning - Tsotsos, Culhane, et al. - 1995
190	Voice Puppetry - Brand - 1999
189	Video Rewrite: Driving Visual Speech with Audio - Bregler, Covell, et al. - 2002
186	Synthesizing Realistic Facial Expressions from Photographs - Pighin, Hecker, et al. - 1998
133	Modeling the Role of Salience in The Allocation of Overt Visual Attention,” Vision Res - Parkhurst, Law, et al. - 2002
110	Trainable Videorealistic Speech Animation - Ezzat, Geiger, et al. - 2002
98	Wiesel: Columnar specificity of intrinsic horizontal and corticocortical connections in cat visual cortex - Gilbert, N - 1989
97	Foundations of Vision. Sinauer Associates - Wandell - 1995
75	Feature-based attention influences motion processing gain in macaque visual cortex - Treue, Trujilo, et al. - 1999
68	Visual cortical mechanisms detecting focal orientation discontinuities - Sillito, Grieve, et al. - 1995
63	The architecture of perceptual spatial interactions - Polat, Sagi - 1994
52	A model of saccade generation based on parallel processing and competitive inhibition - JM, Walker - 1999
51	Animat vision: Active vision in artificial animals. Videre - Terzopoulos, Rabie - 1997
48	Overcomplete steerable pyramid filters and rotation invariance - Greenspan, Belongie, et al. - 1994
45	A vision-based approach to behavioural animation - Renault, Magnenat-Thalmann, et al. - 1990
45	Contrast dependence of contextual effects in primate visual cortex - Levitt, Lund - 1997
42	The guidance of eye movements during active visual search - Motter, Belky - 1998
41	Control of goal-directed and stimulus-driven attention in the brain - Corbetta, Shulman - 2002
41	N.I.: Eyes alive - Lee, Badler, et al. - 2002
39	Computational modelling of visual attention. Nature Reviews Neuroscience - ITTI, KOCH - 1965
38	C: Constraints on cortical and thalamic projections: The no-strong-loops hypothesis. Nature - Crick, Koch - 1998
38	Intrinsic laminar lattice connections in primate visual cortex - Rockland, Lund - 1983
34	Spatial interactions in apparent contrast: Individual differences in enhancement and suppression effects - Cannon, Fullenkamp - 1993
34	A cortico-subcortical model for generation of spatially accurate sequential saccades - Dominey, Arbib - 1992
33	Attentive mechanisms for dynamic and static scene analysis - Milanese, Gil, et al. - 1995
33	Patterns of excitation and inhibition evoked by horizontal connections in visual cortex share a common relationship to orientation columns. Neuron 15:541–552 - Weliky, Kandler, et al. - 1995
32	Two cortical visual systems,” in Analysis of Visual Behavior, eds - Ungerleider, Mishkin - 1982
25	Evaluation of optical motion information by movement detectors. Journal of Comparative Physiology A 161: 533–547 - Reichardt - 1987
23	Neural systems control of spatial orienting - Posner, Cohen, et al. - 1982
23	L.: A goal oriented attention guidance model - Navalpakkam, Itti - 2002
22	Human neural systems for face recognition and social communication - Haxby, Hoffman, et al. - 2002
22	Dynamics of automatic and controlled visual attention - Weichselgartner, Sperling - 1987
22	Estimating focus of attention based on gaze and sound - Stiefelhagen, Yang, et al. - 2001
20	Visual search in continuous, naturalistic stimuli - Wolfe - 1994
16	3-D facial pose and gaze point estimation using a robust real-time tracking paradigm - Heinzmann, Zelinsky - 1998
16	Spatial-frequency selectivity of cells in macaque visual cortex - Valois, Albrecht, et al. - 1982
13	DL: The brainstem control of saccadic eye movements. Nat Rev Neurosci 2002 - Sparks
12	Visual Perception: The Neurophysiological Foundations - SPILLMANN - 1990
10	Gaze estimation using morphable models - Rikert, Jones - 1998
10	Contextual modulation of target saliency - Torralba
9	Synthetic vision and memory for autonomous virtual humans - Peters, O’Sullivan - 2002
8	Perceptual attention in virtual humans: Towards realistic and believable gaze behaviors - Hill