We describe a method for using a database of recorded speech and captured motion to create an animated conversational character. People's utterances are composed of short, clearly-delimited phrases; in each phrase, gesture and speech go together meaningfully and synchronize at a common point of maximum emphasis. We develop tools for collecting and managing performance data that exploit this structure. The tools help create scripts for performers, help annotate and segment performance data, and structure specific messages for characters to use within application contexts. Our animations then reproduce this structure. They recombine motion samples with new speech samples to recreate coherent phrases, and blend segments of speech and motion together phraseby -phrase into extended utterances. By framing problems for utterance generation and synthesis so that they can draw closely on a talented performance, our techniques support the rapid construction of animated characters with rich and appropriate expression.

Predicting when a person might be frustrated can provide an intelligent system with important information about when to initiate interaction. For example, an automated Learning Companion or Intelligent Tutoring System might use this information to intervene, providing support to the learner who is likely to otherwise quit, while leaving engaged learners free to discover things without interruption. This paper presents the first automated method that assesses, using multiple channels of affect-related information, whether a learner is about to click on a button saying ‘‘I’m frustrated.’’ The new method was tested on data gathered from 24 participants using an automated Learning Companion. Their indication of frustration was automatically predicted from the collected data with 79% accuracy (chance 58%). The new assessment method is based on Gaussian process classification and Bayesian inference. Its performance suggests that non-verbal channels carrying affective cues can help provide important information to a system for formulating a more intelligent response.

When building agents and synthetic characters, and in order to achieve believability, we must consider the emotional relations established between users and characters, that is, we must consider the issue of ”empathy”. Defined in broad terms as ”An observer reacting emotionally because he perceives that another is experiencing or about to experience an emotion”, empathy is an important element to consider in the creation of relations between humans and agents. In this paper we will focus on the role of empathy in the construction of synthetic characters, providing some requirements for such construction and illustrating the presented concepts with a specific system called FearNot!. FearNot! was developed to address the difficult and often devastating problem of bullying in schools. By using role playing and empathic synthetic characters in a 3D environment, FearNot! allows children from 8 to 12 to experience a virtual scenario where they can witness (in a third-person perspective) bullying situations. To build empathy into FearNot! we have considered the following components: agent’s architecture; the characters ’ embodiment and emotional expression; proximity with the user and emotionally charged situations. We will describe how these were implemented in FearNot! and report on the preliminary results we have with it.

In this paper, we report on our efforts in developing affective character-based interfaces, i.e. interfaces that recognize and measure affective information of the user and address user affect by employing embodied characters. In particular, we describe the Empathic Companion, an animated interface agent that accompanies the user in the setting of a virtual job interview. This interface application takes physiological data (skin conductance and electromyography) of a user in real-time, interprets them as emotions, and addresses the user's affective states in the form of empathic feedback. We present preliminary results from an exploratory study that aims to evaluate the impact of the Empathic Companion by measuring users' skin conductance and heart rate.

Abstract not found

The design of robust interfaces that process conversational speech is a challenging research direction largely because users’ spoken language is so variable. This research explored a new dimension of speaker stylistic variation by examining whether users’ speech converges systematically with the text-to-speech (TTS) heard from a software partner. To pursue this question, a study was conducted in which twenty-four 7-to-10-yearold children conversed with animated partners that embodied different TTS voices. An analysis of children’s amplitude, durational features, and dialogue response latencies confirmed that they spontaneously adapt several basic acoustic-prosodic features of their speech 10-50%, with the largest adaptations involving utterance pause structure and amplitude. Children’s speech adaptations were relatively rapid, bidirectional, and dynamically readaptable when introduced to new partners, and generalized across different types of users and TTS voices. Adaptations also occurred consistently, with 70-95 % of children converging with their partner’s TTS, although individual differences in magnitude of adaptation were evident. In the design of future conversational systems, users’ spontaneous convergence could be exploited to guide their speech within system processing bounds, thereby enhancing robustness. Adaptive system processing could yield further significant performance gains. The long-term goal of this research is the development of predictive models of human-computer communication to guide the design of new

We studied the role of the competence of an interface agent that helped users to learn and use a text editor. Participants in the study made a set of changes to a document with the aid of one of four interface agents. The agent would respond to participants ’ spoken questions as well as make proactive suggestions using a synthesized voice. The agents varied in the quality of responses and suggestions. One group of participants could also access a help screen. Results revealed that the perceived utility of the agent was influenced by the types of errors it made, while participants ' subjective impressions of the agent related to the perceptions of its embodiment. Additionally, allowing participants to choose their preferred assistance styles improved objective performance. We relate quantitative findings with qualitative interview data and discuss implications for the design and the implementation of systems with interface agents.

Abstract. Pedagogical agent research seeks to exploit Reeves and Nass’s Media Equation, which holds that users respond to interactive media as if they were social actors. Investigations have tended to focus on the media used to realize the pedagogical agent, e.g., the use of animated talking heads and voices, and the results

As an increasing number of new technologies are turning a strong focus on health assessment applications, new engineering and design challenges emerge. Challenges such as inference, modeling, data mining, and feedback for long-term usage arise. This paper argues that embedding empathy into the design of these interactive systems can potentially be vital in the acceptance and success of these types of technologies. This paper discusses three pieces of work that illustrate that designing systems that are intentionally empathetic can play a significant role in creating a better user experience in human-computer interactions. 1.

Abstract. Politeness may play a role in tutorial interaction, including promoting learner motivation and avoiding negative affect. Politeness theory can account for this as a means of mitigating the face threats arising in tutorial situations. It further provides a way of accounting for differences in politeness in different cultures.