Abstract

We describe a computer vision system for observing facial motion by using an optimal estimation optical flow method coupled with geometric, physical and motion-based dynamic models describing the facial structure. Our method produces a reliable parametric representation of the face's independent muscle action groups, as well as an accurate estimate of facial motion. Previous efforts at analysis of facial expression have been based on the Facial Action Coding System (FACS), a representation developed in order to allow human psychologists to code expression from static pictures. To avoid use of this heuristic coding scheme, we have used our computer vision system to probabilistically characterize facial motion and muscle activation in an experimental population, thus deriving a new, more accurate representation of human facial expressions that we call FACS+. Finally, we show how this method can be used for coding, analysis, interpretation, and recognition of facial expressions.

Citations

562	View-based and modular eigenspaces for face recognition. Computer Vision and Pattern Recognition - Pentland, Moghaddam, et al. - 1994
211	Tracking and recognizing rigid and non-rigid facial motion using local parametric models of image motion - Black, Yacoob - 1995
201	Finite Element Procedures in Engineering Analysis - Bathe - 1982
158	Shape and non-rigid motion estimation through physics-based synthesis - Metaxas, Terzopoulos - 1993
135	Layered representation for motion analysis - Wang, Adelson - 1993
116	Animating Facial Expressions - Platt, Badler - 1981
114	Facial action coding system - Ekman, Friesen - 1977
111	3-D motion estimation in model-based facial image coding - LI, ROIVAINEN, et al. - 1993
92	Facial expression recognition using a dynamic model and motion energy - ESSA, PENTLAND - 1995
82	Face recognition using view-based and modular eigenspaces Automatic Systems for the Identification and Inspection of Humans, SPIE - Moghaddam, Pentland - 1994
74	Computing spatio-temporal Representations of Human Faces - Yacoob, Davis - 1994
58	Distributed Representation and Analysis of Visual Motion - Simoncelli - 1993
56	Introduction to Random Signal Analysis and Kalman Filtering - Brown - 1983
55	Modeling, tracking and interactive animation of faces and heads: Using input from video - ESSA, BASU, et al. - 1996
45	Analysis, Interpretation, and Synthesis of Facial Expressions - Essa - 1994
36	Modeling and Animating Faces Using Scanned Data - Waters, Terzopoulos - 1991
36	The argument and evidence about universals in facial expressions of emotion - Ekman - 1989
36	Control System Design : An Introduction To State-Space Methods - Friedland - 1986
35	Interactive graphics for plastic surgery: A task-level analysis and implementation - PIEPER, ROSEN, et al. - 1992
34	Tracking facial motion - Essa, Darrell, et al. - 1994
33	Human emotion recognition from motion using a radial basis function network architecture - Rosenblum, Yacoob, et al. - 1994
27	Facial motion in the perception of faces and of emotional expression - Bassili - 1978
27	Recognizing Faces, Lawrence Erlbaum Associates - Bruce - 1988
22	Analysis and synthesis of facial image sequences using physical and anatomical models - TERZOPOULUS, WATERS - 1993
21	Final report to NSF of the planning workshop on facial expression understanding. Human Interaction - Ekman, Huang, et al. - 1993
16	Closed form solutions for physically based shape modelling and recognition - Pentland, Sclaroff - 1991
15	Lipreading by optical flow - MASE, PENTLAND - 1991
14	Final report to NSF of the standards for facial animation workshop - Pelachaud, Badler, et al. - 1994
13	Recognition of facial expressions for optical flow - Mase - 1991
11	Motion understanding using phase portraits - Shavit, Jepson - 1995
11	The perception of people - Bruner, Tagiuri - 1954
11	A unified approach for physical and geometric modeling for graphics and animation”, proc - Essa, Scarloff, et al. - 1992
2	Emotion recognition: The role of facial motion and the relative importance of upper and lower areas of the face - Bassili - 1979
1	The Society of Mind. A Touchstone Book, Simon and - Minsky - 1985
1	Facial expressionsand the regulation of emotions - Izard - 1990