Smart avatars are virtual human representations controlled by real people. Given instructions interactively, smart avatars can act as autonomous or reactive agents. During a real-time simulation, a user should be able to dynamically refine his or her avatar's behavior in reaction to simulated stimuli without having to undertake a lengthy off-line programming session. In this paper, we introduce an architecture, which allows users to input immediate or persistent instructions using natural language and see the agents' resulting behavioral changes in the graphical output of the simulation. Keywords Autonomous agents, natural language processing, smart avatars, virtual environments. 1. INTRODUCTION In this paper, we describe smart avatars [28, 34], which are virtual representations of humans controlled by real people. Given instructions interactively, smart avatars can act as autonomous or reactive agents. During a real-time simulation, a user should be able to dynamically refine his ...

The process of microplanning encompasses a range of problems in Natural Language Generation (NLG), such as referring expression generation, lexical choice, and aggregation, problems in which a generator must bridge underlying domain-specific representations and general linguistic representations. In this paper, we describe a uniform approach to microplanning based on declarative representations of a generator's communicative intent. These representations describe the RE- SULTS of NLG: communicative intent associates the concrete linguistic structure planned by the generator with inferences that show how the meaning of that structure communicates needed information about some application domain in the current discourse context. Our approach, implemented in the SPUD (sentence planning using description) microplanner, uses the lexicalized treeadjoining grammar formalism (LTAG) to connect structure to meaning and uses modal logic programming to connect meaning to context. At the same time, communicative intent representations provide a RESOURCE for the PROCESS of NLG. Using representations of communicative intent, a generator can augment the syntax, semantics and pragmatics of an incomplete sentence simultaneously, and can assess its progress on the various problems of microplanning incrementally. The declarative formulation of communicative intent translates into a well-defined methodology for designing grammatical and conceptual resources which the generator can use to achieve desired microplanning behavior in a specified domain. Contents 1 Motivation 3 2

The last few years have seen great maturation in understanding how to use computer graphics technology to portray 3D embodied characters or virtual humans. Unlike the off-line, animator-intensive methods used in the special effects industry, real-time embodied agents are expected to exist and interact with us "live." They can be represent other people or function as autonomous helpers, teammates, or tutors enabling novel interactive educational and training applications. We should be able to interact and communicate with them through modalities we already use, such as language, facial expressions, and gesture. Various aspects and issues in real-time virtual humans will be discussed, including consistent parameterizations for gesture and facial actions using movement observation principles, and the representational basis for character believability, personality, and affect. We also describe a Parameterized Action Representation (PAR) that allows an agent to act, plan, and reason about its actions or actions of others. Besides embodying the semantics of human action, the PAR is designed for building future behaviors into autonomous agents and controlling the animation parameters that portray personality, mood, and affect in an embodied agent.

Our goal is to create an “intelligent” 3D agent able to send complex, ‘natural ’ messages to users and, in the future, to converse with them. We look at the relationship between the agent’s communicative intentions and the way that these intentions are expressed into verbal and nonverbal messages. In this paper, we concentrate on the study and generation of coordinated linguistic and gaze communicative acts. In this view we analyse gaze signals according to their functional meaning rather than to their physical actions. We propose a formalism where a communicative act is represented by two elements: a meaning (that corresponds to a set of goals and beliefs that the agent has the purpose to transmit to the interlocutor) and a signal, that is the nonverbal expression of that meaning. We also outline a methodology to generate messages that coordinate verbal with nonverbal signals.

The paper reviews the intersection of AI and VEs. It considers the use of AI as a component of a VE and Intelligent Virtual Agents as a major application area, covering movement, sensing, behaviour and control architectures. It surveys work on emotion and natural language interaction, and considers interactive narrative as a case-study. It concludes by assessing the strengths and weaknesses of the current state-of-the-art and what might take it forward.

This chapter focuses on challenges to improving the realism of socially intelligent agents and attempts to reflect the state of the art in human behavior modeling with particular attention to the impact of values, emotion, and physiology/stress upon individual and group decision-making. The goal is to help those interested in constructing more realistic software agents for use in human performance simulations in both training and analysis settings. The first two sections offer an assessment of the state of the practice and of the need to make better use of human performance moderator functions (PMFs) published in the behavioral literature. The third section pursues this goal by providing an illustrative framework for integrating existing PMF theories and models, such as those on physiology and stress, cognitive and emotive processes, individual differences, and group and crowd behavior, among others. The fourth section presents asymmetric warfare and civil unrest case studies to examine some of the concems affecting implementation of PMFs such as verification, validation, and interoperability with existing simulators, artificial life emulators, and artificial intelligence components. The final section of this chapter concludes with lessons leamed and with some challenges if the field is to reach a greater level of maturity.

This paper presents three experiments designed to empirically evaluate humanoid synthetic agents in electronic retail applications. Firstly, human-like agents were evaluated in a single e-retail application, a home furnishings service. The second experiment explored application dependency effects by evaluating the same human-like agents in a different e-retail application, a personalized CD service. The third experiment evaluated the effectiveness of a range of humanoid cartoon-like agents. Participants eavesdropped on spoken dialogues between a "customer" and each of the agents, which played the role of conversational sales assistants. Results showed participants expected a high level of realistic human-like verbal and nonverbal communicative behavior from the human-like agents. Overall ratings of the agents showed no significant application dependency: Two different groups of participants rated the human-like agents in similar ways in a different application. Further results showed participants have a preference for three--dimensional (3-D) rather than two--dimensional (2-D) cartoon-like agents and have a desire to interact with fully embodied agents. I.

Research in computational linguistics, computer graphics and autonomous agents has led to the development of increasingly sophisticated communicative agents over the past few years, bringing new perspective to machine translation research. The engineering of language-based smooth, expressive, natural-looking human gestures can give us useful insights into the design principles that have evolved in natural communication between people. In this paper we prototype a machine translation system from English to American Sign Language (ASL), taking into account not only linguistic but also visual and spatial information associated with ASL signs.

Humans use gestures in most communicative acts. How are these gestures initiated and performed? What kinds of communicative roles do they play and what kinds of meanings do they convey? How do listeners extract and understand these meanings? Will it be possible to build computerized communicating agents that can extract and understand the meanings and accordingly simulate and display expressive gestures on the computer in such a way that they can be effective conversational partners? All these questions are easy to ask, but far more difficult to answer. In the thesis we try to address these questions regarding the synthesis and acquisition of communicative gestures. Our approach to gesture is...

This paper discusses the representational basis for character believability, personality, and affect. We also describe, a Parameterized Action Representation (PAR) that allows an agent to act, plan, and reason about its actions or actions of others. Besides embodying the semantics of human action, the PAR is designed for building future behaviors into autonomous agents and controlling the animation parameters that portray personality, mood, and affect in an embodied agent