The question is not whether intelligent machines can have any emotions, but whether machines can be intelligent without any emotions. – Marvin Minsky, (Minsky, 1986) p. 163 In every art form it is the emotional content that makes the difference between mere technical skill and true art.

this article, we describe the key areas in which we are extending Steve. Although there has been extensive prior research in each of these areas, we cannot simply plug in different modules representing the state of the art from the different research communities. Researchers develJULY /AUGUST 2002 computer.org/intelligent 33 Figure 1. Steve, an interactive agent that functions as a collaborative instructor or teammate in a virtual world, describes a power light

In recent years, several paradigms have emerged for interactive storytelling. In character-based storytelling, plot generation is based on the behaviour of autonomous characters. In this paper, we describe user interaction in a fully-implemented prototype of an interactive storytelling system. We describe the planning techniques used to control autonomous characters, which derive from HTN planning. The hierarchical task network representing a characters' potential behaviour constitute a target for user intervention, both in terms of narrative goals and in terms of physical actions carried out on stage. We introduce two different mechanisms for user interaction: direct physical interaction with virtual objects and interaction with synthetic characters through speech understanding. Physical intervention exists for the user in on-stage interaction through an invisible avatar: this enables him to remove or displace objects of narrative significance that are resources for character's actions, thus causing these actions to fail. Through linguistic intervention, the user can influence the autonomous characters in various ways, by providing them with information that will solve some of their narrative goals, instructing them to take direct action, or giving advice on the most appropriate behaviour. We illustrate these functionalities with examples of system-generated behaviour and conclude with a discussion of scalability issues.

Emotions have a powerful impact on behavior and beliefs. The goal of our research is to create general computational models of this interplay of emotion, cognition and behavior to inform the design of virtual humans. Here, we address an aspect of emotional behavior that has been studied extensively in the psychological literature but largely ignored by computational approaches, emotion-focused coping. Rather than motivating external action, emotion-focused coping strategies alter beliefs in response to strong emotions. For example an individual may alter beliefs about the importance of a goal that is being threatened, thereby reducing their distress. We present a preliminary model of emotion-focused coping and discuss how coping processes, in general, can be coupled to emotions and behavior. The approach is illustrated within a virtual reality training environment where the models are used to create virtual human characters in high-stress social situations.

The effectiveness of simulation-based training for individual tasks -- such as piloting skills -- is well established, but its use for team training raises challenging technical issues. Ideally, human users could gain valuable leadership experience by interacting with synthetic teammates in realistic and potentially stressful scenarios. However, creating human-like teammates that can support flexible, natural interactions with humans and other synthetic agents requires integrating a wide variety of capabilities, including models of teamwork, models of human negotiation, and the ability to participate in face-to-face spoken conversations in virtual worlds. We have developed such virtual humans by integrating and extending prior work in these areas, and we have applied our virtual humans to an example peacekeeping training scenario to guide and evaluate our research. Our models allow agents to reason about authority and responsibility for individual actions in a team task and, as appropriate, to carry out actions, give and accept orders, monitor task execution, and negotiate options. Negotiation is guided by the agents' dynamic assessment of alternative actions given the current scenario conditions, with the aim of guiding the human user towards an ability to make similar assessments.

This paper describes the virtual humans developed as part of the Mission Rehearsal Exercise project, a virtual reality based training system. This project is an ambitious exercise in integration, both in the sense of integrating technology with entertainment industry content, but also in that we have joined a number of component technologies that have not been integrated before. This integration has not only raised new research issues, but it has also suggested some new approaches to difficult problems. We describe the key capabilities of the virtual humans, including task representation and reasoning, natural language dialogue, and emotion reasoning, and show how these capabilities are integrated to provide more human-level intelligence than would otherwise be possible.

In this paper we introduce ALMA – A Layered Model of Affect. It integrates three major affective characteristics: emotions, moods and personality that cover short, medium, and long term affect. The use of this model consists of two phases: In the preparation phase appraisal rules and personality profiles for characters must be specified with the help of AffectML – our XML based affect modeling language. In the runtime phase, the specified appraisal rules are used to compute real-time emotions and moods as results of a subjective appraisal of relevant input. The computed affective characteristics are represented in AffectML and can be processed by sub-sequent modules that control the cognitive processes and physical behavior of embodied conversational characters. ALMA is part of the VirtualHuman project which develops interactive virtual characters that serve as dialog partners with human-like conversational skills. ALMA provides our virtual humans with a personality profile and with real-time emotions and moods. These are used by the multimodal behavior generation module to enrich the lifelike and believable qualities.

In this paper, we introduce a toolkit called SceneMaker for authoring scenes for adaptive, interactive performances. These performances are based on automatically generated and prescripted scenes which can be authored with the SceneMaker in a two-step approach: In step one, the scene flow is defined using cascaded finite state machines. In a second step, the content of each scene must be provided. This can be done either manually by using a simple scripting language, or by integrating scenes which are automatically generated at runtime based on a domain and dialogue model. Both scene types can be interweaved in our planbased, distributed platform. The system provides a context memory with access functions that can be used by the author to make scenes user-adaptive. Using CrossTalk as the target application, we describe our models and languages, and illustrate the authoring process. CrossTalk is an interactive installation with animated presentation agents which “live ” beyond the actual presentation and systematically step out of character within the presentation, both to enhance the illusion of life. The context memory enables the system to adapt to user feedback and generates data for later evaluation of user/system behavior. The SceneMaker toolkit should enable the non-expert to compose adaptive, interactive performances in a rapid prototyping approach.

Head pose and gesture offer several key conversational grounding cues and are used extensively in face-to-face interaction among people. We investigate how dialog context from an embodied conversational agent (ECA) can improve visual recognition of user gestures. We present a recogntion framework which (1) extracts contextual features from an ECA´s dialog manager, (2) computes a predicition of head nod and head shakes, and (3) integrates the contextual predictions with the visual observation of a vision-based head gesture recognizer. We found a subset of lexical, punctuation and timing features that are easily available in most ECA architectures and can be used to learn how to predict user feedback. Using a discriminative approach to contextual prediction and multi-modal integration, we were able to improve the performancae of head gesture detection even when the topic of the test set was significantly different than the training set.

Embodied conversational agents provide a promising option for presenting information to users. This contribution revisits a number of past and ongoing systems with animated characters that have been developed at DFKI. While in all systems the purpose of using characters is to convey information to the user, there are significant variations in the style of presentation and the assumed conversational setting. The spectrum of systems include systems that feature a single, TV-style presentation agent, dialogue systems, as well as systems that deploy multiple interactive characters. We also provide a technical view on these systems and sketch the underlying system architectures of each sample system.