This paper explores the use of local parametrizedmodels of image motion for recovering and recognizing the non-rigid and articulated motion of human faces. Parametric flow models (for example affine) are popular for estimating motion in rigid scenes. We observe that within local regions in space and time, such models not only accurately model non-rigid facial motions but also provide a concise description of the motion in terms of a small number of parameters. These parameters are intuitively related to the motion of facial features during facial expressions and we show how expressions such as anger, happiness, surprise, fear, disgust, and sadness can be recognized from the local parametric motions in the presence of significant head motion. The motion tracking and expression recognition approach performs with high accuracy in extensive laboratory experiments involving 40 subjects as well as in television and movie sequences. 1 Introduction This paper describes a new...

This paper surveys the past work in solving these problems. The capability of the human visual system with respect to these problems is discussed, too. It is meant to serve as an ultimate goal and a guide for determining recommendations for development of an automatic facial expression analyzer

Pinker, S. & Bloom, P. (1990). Natural language and natural selection. Behavioral and Brain Sciences 13

This paper explores the use of local parametrized models of image motion for recovering and recognizing the non-rigid and articulated motion of human faces. Parametric flow models (for example affine) are popular for estimating motion in rigid scenes. We observe that within local regions in space and time, such models not only accurately model non-rigid facial motions but also provide a concise description of the motion in terms of a small number of parameters. These parameters are intuitively related to the motion of facial features during facial expressions and we show how expressions such as anger, happiness, surprise, fear, disgust, and sadness can be recognized from the local parametric motions in the presence of significant head motion. The motion tracking and expression recognition approach performed with high accuracy in extensive laboratory experiments involving 40 subjects as well as in television and movie sequences.

An approach for analysis and representation of facial dynamics for recognition of facial expressions from image sequences is proposed. The algorithms we develop utilize optical flow computation to identify the direction of rigid and non-rigid motions that are caused by human facial expressions. A mid-level symbolic representation that is motivated by linguistic and psychological considerations is developed. Recognition of six facial expressions, as well as eye blinking, on a large set of image sequences is reported. 1 Introduction Human visual communication has been extensively studied in the social and psychology literature, mainly as a means to describe the emotional state of the subject [3,5,11,15]. Research in psychology has indicated that at least six emotions are universally associated with distinct facial expressions. Several other emotions, and many combinations of emotions, have been studied but remain unconfirmed as universally distinguishable. The six principle emotions are...

This paper discusses our expert system called Integrated System for Facial Expression Recognition (ISFER), which performs recognition and emotional classification of human facial expression from a still full-face image. The system consists of two major parts. The first one is the ISFER Workbench, which forms a framework for hybrid facial feature detection. Multiple feature detection techniques are applied in parallel. The redundant information is used to define unambiguous face geometry containing no missing or highly inaccurate data. The second part of the system is its inference engine called HERCULES, which converts low level face geometry into high level facial actions, and then this into highest level weighted emotion labels.

Emotions are universally recognized from facial expressions—or so it has been claimed. To support that claim, research has been carried out in various modern cultures and in cultures relatively isolated from Western influence. A review of the methods used in that research raises questions of its ecological, convergent, and internal validity. Forced-choice response format, within-subject design, preselected photographs of posed facial expressions, and other features of method are each problematic. When they are altered, less supportive or nonsupportive results occur. When they are combined, these method factors may help to shape the results. Facial expressions and emotion labels are probably associated, but the association may vary with culture and is loose enough to be consistent with various alternative accounts, 8 of which are discussed. "Everyone knows that grief involves a gloomy and joy a cheerful countenance.... There are characteristic facial expressions which are observed to accompany anger, fear, erotic excitement, and all the other passions " (Aristotle, nd/1913, pp. 805, 808). Aristotle was not proposing a new idea but was cataloging what was known on the topic of physiognomy. The theory was that a person's physical appearance, especially in the face, reveals deeper characteristics: Poor proportions reveal a rogue, soft hair a coward, and a smile a happy person.' Today, few psychologists share Aristotle's belief about the meaning of poor proportions or soft hair, but many share his beliefs about facial expression and emotion. Oatley and Jenkins (1992) observed, "By far the most extensive body of data in the field of human emotions is that on facial expressions of emotion" (p. 67). Recent reviews of those data (see Table 1) agree that the face reveals emotion in a way that is universally understood: Happiness, surprise, fear, anger, contempt, disgust, and sadness—these seven emotions, plus or minus two, are recognized from facial expressions by all human beings, regardless of their cultural background.

Abstract. We describe an Affect and Belief Adaptive Interface System (ABAIS) designed to compensate for performance biases caused by users ’ affective states and active beliefs. The ABAIS architecture implements an adaptive methodology consisting of four steps: sensing/inferring user affective state and performance-relevant beliefs; identifying their potential impact on performance; selecting a compensatory strategy; and implementing this strategy in terms of speci¢c GUI adaptations. ABAIS provides a generic adaptive framework for integrating a variety of user assessment methods (e.g. knowledge-based, self-reports, diagnostic tasks, physiological sensing), and GUI adaptation strategies (e.g. content- and format-based). The ABAIS performancebias prediction isbased on empirical ¢ndings from emotion research combined with detailed knowledge of the task context.The initial ABAIS prototype was demonstrated in the context of an Air Force combat task, used a knowledge-based approach to assess the pilot’s anxiety level, and adapted to the pilot’s anxiety and belief states by modifying selected cockpit instrument displays in response to detected changes in these states.

cation as an integrated set of speech, gaze, and gestural behaviours. As we shall see, studies of mediated communication allow us to identify the contribution of different nonverbal behaviours, such as gaze and gesture to communication. They also help to clarify the overall influence of visual information in communication, and the effects of communication interactive processes such as feedback on communication. Prior to examining what research has to say about these issues, we describe: (a) mediated communication theories; (b) the technologies used to support communication in the various situations; (c) the methods used to study mediated communication. Theories of mediated communication There are many different theories of mediated communication. Rather than describing each in detail, we restrict ourselves here to an exposition of the general form that these theories take, and the general set of claims that they make. The fundamental goal of mediated communication theor

We propose a fuzzy rule-based system to map representations of the emotional state of an animated agent onto muscle contraction values for the appropriate facial expressions. Our implementation pays special attention to the way in which continuous changes in the intensity of emotions can be displayed smoothly on the graphical face. The rule system we have defined implements the patterns described by psychologists and researchers dealing with facial expressions of humans, including rules for displaying blends of expressions.