The question is not whether intelligent machines can have any emotions, but whether machines can be intelligent without any emotions. – Marvin Minsky, (Minsky, 1986) p. 163 In every art form it is the emotional content that makes the difference between mere technical skill and true art.

The ability to recognize affective states of a person... This paper argues that next-generation human-computer interaction (HCI) designs need to include the essence of emotional intelligence -- the ability to recognize a user's affective states -- in order to become more human-like, more effective, and more efficient. Affective arousal modulates all nonverbal communicative cues (facial expressions, body movements, and vocal and physiological reactions). In a face-to-face interaction, humans detect and interpret those interactive signals of their communicator with little or no effort. Yet design and development of an automated system that accomplishes these tasks is rather difficult. This paper surveys the past work in solving these problems by a computer and provides a set of recommendations for developing the first part of an intelligent multimodal HCI -- an automatic personalized analyzer of a user's nonverbal affective feedback.

Automated analysis of human affective behavior has attracted increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and related disciplines. However, the existing methods typically handle only deliberately displayed and exaggerated expressions of prototypical emotions, despite the fact that deliberate behavior differs in visual appearance, audio profile, and timing from spontaneously occurring behavior. To address this problem, efforts to develop algorithms that can process naturally occurring human affective behavior have recently emerged. Moreover, an increasing number of efforts are reported toward multimodal fusion for human affect analysis, including audiovisual fusion, linguistic and paralinguistic fusion, and multicue visual fusion based on facial expressions, head movements, and body gestures. This paper introduces and surveys these recent advances. We first discuss human emotion perception from a psychological perspective. Next, we examine available approaches for solving the problem of machine understanding of human affective behavior and discuss important issues like the collection and availability of training and test data. We finally outline some of the scientific and engineering challenges to advancing human affect sensing technology.

Emotions are universally recognized from facial expressions—or so it has been claimed. To support that claim, research has been carried out in various modern cultures and in cultures relatively isolated from Western influence. A review of the methods used in that research raises questions of its ecological, convergent, and internal validity. Forced-choice response format, within-subject design, preselected photographs of posed facial expressions, and other features of method are each problematic. When they are altered, less supportive or nonsupportive results occur. When they are combined, these method factors may help to shape the results. Facial expressions and emotion labels are probably associated, but the association may vary with culture and is loose enough to be consistent with various alternative accounts, 8 of which are discussed. "Everyone knows that grief involves a gloomy and joy a cheerful countenance.... There are characteristic facial expressions which are observed to accompany anger, fear, erotic excitement, and all the other passions " (Aristotle, nd/1913, pp. 805, 808). Aristotle was not proposing a new idea but was cataloging what was known on the topic of physiognomy. The theory was that a person's physical appearance, especially in the face, reveals deeper characteristics: Poor proportions reveal a rogue, soft hair a coward, and a smile a happy person.' Today, few psychologists share Aristotle's belief about the meaning of poor proportions or soft hair, but many share his beliefs about facial expression and emotion. Oatley and Jenkins (1992) observed, "By far the most extensive body of data in the field of human emotions is that on facial expressions of emotion" (p. 67). Recent reviews of those data (see Table 1) agree that the face reveals emotion in a way that is universally understood: Happiness, surprise, fear, anger, contempt, disgust, and sadness—these seven emotions, plus or minus two, are recognized from facial expressions by all human beings, regardless of their cultural background.

A meta-analysis examined emotion recognition within and across cultures. Emotions were universally recognized at better-than-chance levels. Accuracy was higher when emotions were both expressed and recognized by members of the same national, ethnic, or regional group, suggesting an in-group advantage. This advantage was smaller for cultural groups with greater exposure to one another, measured in terms of living in the same nation, physical proximity, and telephone communication. Majority group members were poorer at judging minority group members than the reverse. Cross-cultural accuracy was lower in studies that used a balanced research design, and higher in studies that used imitation rather than posed or spontaneous emotional expressions. Attributes of study design appeared not to moderate the size of the in-group advantage. Psychologists have long debated whether emotions are universal versus whether they vary by culture. These issues have been extensively summarized elsewhere and we do not reiterate them

H. A. Elfenbein and N. Ambady (2002) examined the evidence for an in-group advantage in emotion recognition, whereby recognition is generally more accurate for perceivers from the same cultural group as emotional expressors. D. Matsumoto’s (2002) comment centered on 3 asserted methodological requirements. This response addresses the lack of consensus concerning these “requirements ” and demonstrates that none alter the presence of the in-group advantage. His analyses had a serious flaw and, once corrected, replicated the original findings. Furthermore, he described results from his empirical work not meeting a literal interpretation of his own requirements. Overall, where Matsumoto considers subtle cross-cultural differences in emotional expression a methodological artifact in judgment studies, the present authors find a core phenomenon worthy of attention. In the current issue of Psychological Bulletin, we presented a meta-analysis providing evidence for both the universality and cultural specificity of emotion recognition (Elfenbein & Ambady, 2002). Matsumoto (2002) commented on one particular finding in this meta-analysis—that there appears to be an in-group advantage in emotion whereby emotional communication is generally more

Face perception and emotion recognition were investigated in a group of people with Huntington's disease and matched controls. In conventional tasks intended to explore the perception of age, sex, unfamiliar face identity (Benton test) and gaze direction from the face, the Huntington 's disease group showed a borderline impairment of gaze direction perception and were significantly impaired on unfamiliar face matching. With a separate set of tasks using computerinterpolated ('morphed') facial images, people with Huntington's disease were markedly impaired at discriminating anger from fear, but experienced less difficulty with continua varying from male to female, between familiar identities, and from happiness to sadness. In a further test of recognition of facial expressions of basic emotions from the Ekman and Friesen (1976) series, interpolated images were created for six continua that lay around the perimeter of an emotion hexagon (happiness-surprise;

This paper reviews processing of facial identity and expressions. The issue of independence between or two systems for these tasks has been addressed from different approaches over the past twenty-five years. More recently, neuroimaging techniques have provided researchers with new tools to investigate how facial information is processed in the brain. First, findings from "traditional" approaches to identity and expression processing are summarized. The review then

This paper includes a proposed model for cultural characters based on the PSI model of emotions combined with Hofstede’s cultural dimensions and the Big Five model of personality. It starts with an introduction to the research, followed by a discussion on the main features of PSI and Hofstede’s cultural model along with the Big Five personality parameters. Finally it discusses the integration of the three models.

ABSTRACT While recognizing the theoretical importance of context, current research has treated naming as though semantic meaning were invariant and the same mapping of category exemplars and names should exist across experimental contexts. An assumed symmetry or bidirectionality in naming behavior has been implicit in the interchangeable use of tasks that ask subjects to match names to stimuli and tasks that ask subjects to match stimuli to names. Examples from the literature are discussed together with several studies of color naming and basic emotion naming in which no such symmetry was found. A more complete model of naming is proposed to account for flexible mapping of names to items. Principles of naming are suggested to describe effects of stimulus sampling, differing access to terms, task demands, and other impacts on naming behavior. 1.