Recent years have witnessed the birth of a new paradigm for learning environments: animated pedagogical agents. These lifelike autonomous characters cohabit learning environments with students to create rich, face-to-face learning interactions. This opens up exciting new possibilities; for example, agents can demonstrate complex tasks, employ locomotion and gesture to focus students'attention on the most salient aspect of the task at hand, and convey emotional responses to the tutorial situation. Animated pedagogical agents offer great promise for broadening the bandwidth of tutorial communication and increasing learning environments' ability to engage and motivate students. This article sets forth the motivations behind animated pedagogical agents, describes the key capabilities they offer, and discusses the technical issues they raise. The discussion is illustrated with descriptions of a number of animated agents that represent the current state of the art.

To explain complex phenomena, an explanation system must be able to select information from a formal representation of domain knowledge, organize the selected information into multisentential discourse plans, and realize the discourse plans in text. Although recent years have witnessed significant progress in the development of sophisticated computational mechanisms for explanation, empirical results have been limited. This paper reports on a seven-year effort to empirically study explanation generation from semantically rich, large-scale knowledge bases. In particular, it describes KNIGHT, a robust explanation system that constructs multisentential and multiparagraph explanations from the Biology Knowledge Base, a large-scale knowledge base in the domain of botanical anatomy, physiology, and development. We introduce the Two-Panel evaluation methodology and describe how KNIGHT'S performance was assessed with this methodology in the most extensive empirical evaluation conducted on an explanation system. In this evaluation, KNIGHT scored within "half a grade " of domain experts, and its performance exceeded that of one of the domain experts. 1.

We are working toward animated agents that can collaborate with human students in virtual worlds. The agent's objective is to help students learn to perform physical, procedural tasks, such as operating and maintaining equipment. Like most of the previous research on task-oriented dialogues, the agent (computer) serves as an expert that can provide guidance to a human novice. Research on such dialogues dates back more than twenty years (Deutsch 1974), and the subject remains an active research area (Allen et al. 1996; Lochbaum 1994; Walker 1996). However, most of that research has focused solely on verbal dialogues, even though the earliest studies clearly showed the ubiquity of nonverbal communication in human task-oriented dialogues (Deutsch 1974). To allow a wider variety of interactions among agents and human students, we use virtual reality (Durlach and Mavor 1995); agents and students cohabit a threedimensional, interactive, simulated mock-up of the student'

Abstract. Mixed-initiative problem solving lies at the heart of knowledge-based learning environments. While learners are actively engaged in problem-solving activities, learning environments should monitor their progress and provide them with feedback in a manner that contributes to achieving the twin goals of learning effectiveness and learning efficiency. Mixed-initiative interactions are particularly critical for constructivist learning environments in which learners participate in active problem solving. We have recently begun to see the emergence of believable agents with lifelike qualities. Featured prominently in constructivist learning environments, lifelike pedagogical agents could couple key feedback functionalities with a strong visual presence by observing learners ’ progress and providing them with visually contextualized advice during mixed-initiative problem solving. For the past three years, we have been engaged in a large-scale research program on lifelike pedagogical agents and their role in constructivist learning environments. In the resulting computational framework, lifelike pedagogical agents are specified by (1) a behavior space containing animated and vocal behaviors, (2) a design-centered context model that maintains constructivist problem representations, multimodal advisory contexts, and evolving problem-solving tasks, and (3) a behavior sequencing engine that in realtime dynamically selects and assembles agents ’ actions to create pedagogically effective, lifelike behaviors. To empirically investigate this framework, it has been instantiated in a full-scale implementation of a lifelike pedagogical agent for DESIGN-A-PLANT, a learning environment developed for the domain of botanical anatomy and physiology for middle school students. Experience with focus group studies conducted with middle school students interacting with the implemented agent suggests that lifelike pedagogical agents hold much promise for mixed-initiative learning. Key words: Lifelike agents, pedagogicalagents, animated agents, knowledge-basedlearning environments, mixed-initiative interaction, intelligent tutoring systems, intelligent multimedia presentation,

We focus on the production of efficient descriptions of objects, actions and events. We define a type of efficiency, textual economy, that exploits the hearer's recognition of inferential links to material elsewhere within a sentence. Textual economy leads to efficient descriptions because the material that supports such inferences has been included to satisfy independent communicative goals, and is therefore overloaded in the sense of Pollack [18]. We argue that achieving textual economy imposes strong requirements on the representation and reasoning used in generating sentences. The representation must support the generator's simultaneous consideration of syntax and semantics. Reasoningmust enable the generator to assess quickly and reliably at any stage how the hearer will interpret the current sentence, with its '-(inc6mplete)syntax and'semantics. We show that these representational and reasoning requirements are met in the SPUD system for sentence planning and realization.

This paper describes PEBA-II, a working natural language generation system which interactively describes animals in a taxonomic knowledge base via the production of World Wide Web pages. Our aim is to construct a natural language document generation system with real practical applicability: to this end, the system reconstructs and combines a number of existing ideas in the literature in a novel way, and proposes a solution to the problem of breadth of coverage that is based on a pragmatic approach to knowledge representation and linguistic realisation. The system embodies the following features: ffl a reconstruction of some of the core ideas in schema--based text generation [McKeown 1985], applied to the generation of hypertext documents; ffl the principled use of a phrasal lexicon to ease surface generation, in concert with a knowledge base whose elements may correspond to pre--compiled collections of atomic units; ffl a user model and discourse model that permit interesting varia...

We describe the generation of communicative actions in an implemented embodied conversational agent. Our agent plans each utterance so that mul- tiple communicative goals may be realized opportunistically by a composite action including not only speech but also coverbat gesture that fits the con- text and the ongoing speech in ways representative of natural human conversation. We accomplish this by reasoning from a grammar which describes ges- ture declaratively in terms of its discourse function, semantics and synchrony with speech.

Abstract. Lifelike animated agents for knowledge-based learning environments can provide timely, customized advice to support learners ’ problem-solving activities. By drawing on a rich repertoire of emotive behaviors to exhibit contextually appropriate facial expressions and emotive gestures, these agents could exploit the visual channel to more effectively communicate with learners. To address these issues, this article proposes the emotive-kinesthetic behavior sequencing framework for dynamically sequencing lifelike pedagogical agents ’ full-body emotive expression. By exploiting a rich behavior space populated with emotive behaviors and structured by pedagogical speech act categories, a behavior sequencing engine operates in realtime to select and assemble contextually appropriate expressive behaviors. This framework has been implemented in a lifelike pedagogical agent, COSMO, who exhibits full-body emotive behaviors in response to learners ' problem-solving activities.

Lifelike animated agents for knowledge-based learning environments can provide timely, cus-tomized advice to support students ' problem solving. Because of their strong visual presence, they hold signi cant promise for substantially increasing students ' enjoyment of their learning experiences. Akey problem posed by lifelike agents that inhabit arti cial worlds is deictic believability. In the same manner that humans refer to objects in their environment through judicious combinations of speech, locomotion, and gesture, animated agents should be able to move through their environment, and point to and refer to objects appropriately as they provide problem-solving advice. In this paper we describe a framework for achieving deictic believabil-ity in animated agents. A deictic behavior planner exploits a world model and the evolving explanation plan as it selects and coordinates locomotive, gestural, and speech behaviors. The resulting behaviors and utterances are believable, and the references exhibit a lack ofambiguity. This approach to spatial deixis has been implemented in a lifelike animated agent, Cosmo, who inhabits a learning environment for the domain of Internet packet routing. Cosmo provides realtime advice to students as they escort packets through a virtual world of interconnected routers. Results of an informal focus group study with the Cosmo agent suggest that the spatial deixis framework produces clear explanatory animated behaviors. 1 1

In this paper, we present a statistical approach for dialogue act processing in the dialogue component of the speech-to-speech translation system VERBMOBIL. Statistics in dialogue processing is used to predict follow-up dialogue acts. As an application example we show how it supports repair when unexpected dialogue states occur.