The ability to understand and manage social signals of a person we are communicating with is the core of social intelligence. Social intelligence is a facet of human intelligence that has been argued to be indispensable and perhaps the most important for success in life. This paper argues that next-generation computing needs to include the essence of social intelligence – the ability to recognize human social signals and social behaviours like turn taking, politeness, and disagreement – in order to become more effective and more efficient. Although each one of us understands the importance of social signals in everyday life situations, and in spite of recent advances in machine analysis of relevant behavioural cues like blinks, smiles, crossed arms, laughter, and similar, design and development of automated systems for Social Signal Processing (SSP) are rather difficult. This paper surveys the past efforts in solving these problems by a computer, it summarizes the relevant findings in social psychology, and it proposes a set of recommendations for enabling the development of the next generation of socially-aware computing.

A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this prediction is to come true, then next generation computing, which we will call human computing, should be about anticipatory user interfaces that should be human-centered, built for humans based on human models. They should transcend the traditional keyboard and mouse to include natural, human-like interactive functions including understanding and emulating certain human behaviors such as affective and social signaling. This article discusses a number of components of human behavior, how they might be integrated into computers, and how far we are from realizing the front end of human computing, that is, how far are we from enabling computers to understand human behavior.

In this article, I introduce an emotion paradox: People believe that they know an emo-tion when they see it, and as a consequence assume that emotions are discrete events that can be recognized with some degree of accuracy, but scientists have yet to pro-duce a set of clear and consistent criteria for indicating when an emotion is present and when it is not. I propose one solution to this paradox: People experience an emo-tion when they conceptualize an instance of affective feeling. In this view, the experi-ence of emotion is an act of categorization, guided by embodied knowledge about emotion. The result is a model of emotion experience that has much in common with the social psychological literature on person perception and with literature on em-bodied conceptual knowledge as it has recently been applied to social psychology. Humans experience emotion. For many, experience serves as an emotion’s central and defining aspect. We feel the heat of anger, the despair of sadness, the dread of fear. Most days, at least in North America, each of us asks and answers about our emotional state. We talk

This article addresses the debate between emotion-expression andmotive-communica-tion approaches to facial movements, focusing on Ekman's (1972) and Fridlund's (1994) contrasting models and their historical antecedents. Available evidence sug-gests that thepresence ofothers either reduces or increasesfacial responses, depending on the quality and strength ofthe emotional manipulation and on the nature ofthe rela-tionship between interactants. Although both display rules and social motives provide viable explanations ofaudience "inhibition " effects, some audiencefacilitation effects are less easily accommodated within an emotion-expression perspective. In particular emotion is not a sufficient conditionfor a corresponding "expression, " even discount-ing explicit regulation, and, apparently, "spontaneous "facial movements may befacil-itated by thepresence ofothers. Further, there is no direct evidence that any particular facial movementprovides an unambiguous expression ofa specific emotion. However, information communicated byfacial movements is not necessarily extrinsic to emotion. Facial movements not only transmit emotion-relevant information but also contribute to ongoing processes of emotional action in accordance with pragmatic theories. What's in a smile, a scowl, a grimace? People attach a variety of meanings to faces. Some of their attribu-tions accurately reflect what another person is actually feeling, thinking, or doing, whereas other inferences are of more dubious validity. For example, we are ca-pable of detecting quite precisely the direction of someone's visual attention (e.g., Kobayashi & Kohshima, 1997; Rutter, 1987) and register at an im-plicit level how closely attuned their facial movements are to those of others, including ourselves (e.g.,

ABSTRACT: Three experiments were conducted to examine whether the temporal dynamics of Duchenne-smiles influenced the perception of smile authenticity. Realistic computer-generated Duchenne-smiles that varied in their onset-and offset-durations (Experiment 1), or only in their offset-duration (Experiment 2), or in both their onset-and apex-durations (Experiment 3), were created using Poser 4 software. Perceived genuineness varied monotonically with the duration of each manipulated dynamic component. The results are in accordance with

Social Signal Processing is the research domain aimed at bridging the social intelligence gap between humans and machines. This article is the first survey of the domain that jointly considers its three major aspects, namely modeling, analysis and synthesis of social behaviour. Modeling investigates laws and principles underlying social interaction, analysis explores approaches for automatic understanding of social exchanges recorded with different sensors, and synthesis studies techniques for the generation of social behaviour via various forms of embodiment. For each of the above aspects, the paper includes an extensive survey of the literature, points to the most important publicly available resources, and outlines the most fundamental challenges ahead.

This research provides a systematic analysis of the nonverbal expression of pride. Study 1 manipulated behavioral movements relevant to pride (e.g., expanded posture and head tilt) to identify the most prototypical pride expression and determine the specific components that are necessary and sufficient for reliable recognition. Studies 2 and 3 tested whether the 2 conceptually and empirically distinct facets of pride (“authentic ” and “hubristic”; J. L. Tracy & R. W. Robins, 2007a) are associated with distinct nonverbal expressions. Results showed that neither the prototypical pride expression nor several recognizable variants were differentially associated with either facet, suggesting that for the most part, authentic and hubristic pride share the same signal. Together these studies indicate that pride can be reliably assessed from nonverbal behaviors. In the Appendix, the authors provide guidelines for a pride behavioral coding scheme, akin to the Emotion Facial Action Coding System (EMFACS; P. Ekman & E. Rosenberg, 1997) for assessing “basic ” emotions from observable nonverbal behaviors.

A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. To realise this prediction, next-generation computing should develop anticipatory user interfaces that are human-centred, built for humans and based on naturally occurring multimodal human communication. These interfaces should transcend the traditional keyboard and mouse and have the capacity to understand and emulate human communicative intentions as expressed through behavioural cues, such as affective and social signals. This article discusses how far we are to the goal of human-centred computing and Human-Centred Intelligent Human-Computer Interaction (HCI²) that can understand and respond to multimodal human communication.

People differ in the extent to which their verbal reports of experienced emotion are valence focused or arousal focused. Three multimethod studies are reported to explore whether differential focus reflects individual differences in the cognitive structure of emotion language versus differences in phenomeno-logical experience. Although there was some evidence that valence focus and arousal focus were linked to variations differences in cognitive structure, the findings are also consistent with the view that self-report ratings are being driven by the properties of the feelings that are being reported. Implications for the study of experienced emotion are discussed. People report how they feel using words. In everyday life, when we want to know how someone is feeling, we ask. “How are you feeling? ” is one of the most ubiquitous questions in our culture. Likewise, for over half a century, psychologists have relied on self-reports to find out how people feel, and for good reason. Decades of behavioral, cognitive, and psychophysiological re-search have failed to provide clear and consistent measures that unambiguously correspond to the categories of experienced emo-

“The past is never dead. It’s not even past.”