What kinds of social relationships can people have with computers? Are there activities that computers can engage in that actively draw people into relationships with them? What are the potential benefits to the people who participate in these human-computer relationships? To address these questions this work introduces a theory of Relational Agents, which are computational artifacts designed to build and maintain long-term, social-emotional relationships with their users. These can be purely software humanoid animated agents--as developed in this work--but they can also be non-humanoid or embodied in various physical forms, from robots, to pets, to jewelry, clothing, hand-helds, and other interactive devices. Central to the notion of relationship is that it is a persistent construct, spanning multiple interactions; thus, Relational Agents are explicitly designed to remember past history and manage future expectations in their interactions with users. Finally, relationships are fundamentally social and emotional, and detailed knowledge of human social psychology--with a particular emphasis on the role of affect--must be incorporated into these agents if they are to effectively leverage the mechanisms of human social cognition in order to build relationships in the most natural manner possible. People build

In this study, we investigated people's relationships with AIBO, a robotic pet, through 6,438 spontaneous postings in online AIBO discussion forums. Results showed that AIBO psychologically engaged this group of participants, particularly by drawing forth conceptions of technological essences (75%), life-like essences (49%), mental states (60%), and social rapport (59%). However, participants seldom attributed moral standing to AIBO (e.g., that AIBO deserves respect, has rights, or can be held morally accountable for action). Our discussion focuses on how robotic pets (now and in the future) may (a) challenge traditional boundaries (e.g. between who or what can possess feelings), (b) extend our conceptions of self, companionship, and community, and (c) begin to replace interactions with live pets. We also discuss a concern that people in general, and children in particular, may fall prey to accepting robotic pets without the moral responsibilities (and moral developmental outcomes) that real, reciprocal companionship and cooperation involves. This research contributes to a growing literature on the human-robotic relationship.

This paper presents an experiment designed to investigate the impact of visual and behavioral realism in avatars on perceived quality of communication in an immersive virtual environment.

Many problems in vision involve the prediction of a class label for each frame in an unsegmented sequence. In this paper, we develop a discriminative framework for simultaneous sequence segmentation and labeling which can capture both intrinsic and extrinsic class dynamics. Our approach incorporates hidden state variables which model the sub-structure of a class sequence and learn dynamics between class labels. Each class label has a disjoint set of associated hidden states, which enables efficient training and inference in our model. We evaluated our method on the task of recognizing human gestures from unsegmented video streams and performed experiments on three different datasets of head and eye gestures. Our results demonstrate that our model compares favorably to Support Vector Machines, Hidden Markov Models, and Conditional Random Fields on visual gesture recognition tasks.

Abstract — Eyes play a central role in human-human communication, for example, in directing attention and regulating turntaking. For this reason, the eyes have been a central topic in several fields of interaction study. Although many psychological findings have encouraged previous work in both human-computer and human-robot interaction studies, there have been few explorations from the viewpoint of the timing of gaze behavior. In this study, the impression a person forms from an interaction is regarded to be strongly influenced by the feeling of being looked at which is assumed to be based on the responsiveness of the other’s gaze to the person’s one and be the basis of impression conveyance as a communicative being. In this paper, we built a robot that could move its gaze responsively to its interaction partner’s one to explore the effect of responsive gaze. In this paper, we evaluated two primitive ways of controlling a robot’s gaze responsively to its partner and confirmed that robots with such responsive gaze could give stronger feeling of being looked at than ones with non-responsive gaze. I.

Abstract. Gaze plays a large number of cognitive, communicative and affective roles in face-to-face human interaction. To build a believable virtual human, it is imperative to construct a gaze model that generates realistic gaze behaviors. However, it is not enough to merely imitate a person's eye movements. The gaze behaviors should reflect the internal states of the virtual human and users should be able to derive them by observing the behaviors. In this paper, we present a gaze model driven by the cognitive operations; the model processes the virtual human's reasoning, dialog management, and goals to generate behaviors that reflect the agent’s inner thoughts. It has been implemented in our virtual human system and operates in real-time. The gaze model introduced in this paper was originally designed and developed by Jeff Rickel but has since been extended by the authors. 1

Abstract. We present IGaze, a semi-immersive human-avatar interaction system. Using head tracking and an illusionistic 3D effect we let users interact with a talking avatar in an application interview scenario. The avatar features reactive gaze behavior that adapts to the user position according to exchangeable gaze strategies. In user studies we showed that two gaze strategies successfully convey the intended impression of dominance/submission and that the 3D effect was positively received. We argue that IGaze is a suitable setup for exploring reactive nonverbal behavior synthesis in human-avatar interactions. 1

Gaze is an extremely important aspect of human face to face interaction. Over the course of an interaction, a single individual’s gaze can perform many different functions, such as regulating communication, expressing emotion, and attending to task performance. When gaze shifts occur, where they are directed, and how they are performed all provide critical information to an observer of the gaze shift. The goal of this work is to allow virtual humans to mimic the gaze capabilities of humans in face to face interaction. This paper introduces the SmartBody Gaze Controller (SBGC), a highly versatile framework for realizing various manners of gaze through a rich set of input parameters. Using these parameters, the SBCG controls aspects of movement such as velocity, postural bias, and the selection of joints committed to a particular gaze task. We provide a preliminary implementation that demonstrates how related work on the Expressive Gaze Model (EGM) can be used to inform management of these input parameters. The EGM is a model for manipulating the style of gaze shifts for the purpose of expressing emotion [11]. The SBGC is fully compatible with all aspects of the SmartBody system [23].

This paper proposes a gaze-communicative stuffed-toy robot system with joint attention and eye-contact reactions based on ambient gaze-tracking. For free and natural interaction, we adopted our remote gaze-tracking method. Corresponding to the user’s gaze, the gaze-reactive stuffed-toy robot is designed to gradually establish 1) joint attention using the direction of the robot’s head and 2) eye-contact reactions from several sets of motion. From both subjective evaluations and observations of the user’s gaze in the demonstration experiments, we found that i) joint attention draws the user’s interest along with the user-guessed interest of the robot, ii) “eye contact ” brings the user a favorable feeling for the robot, and iii) this feeling is enhanced when “eye contact ” is used in combination with “joint attention.” These results support the approach of our embodied gazecommunication model.

Currently, state of the art virtual agents lack the ability to display emotion as seen in actual humans, or even in hand-animated characters. One reason for the emotional inexpressiveness of virtual agents is the lack of emotionally expressive gaze manner. For virtual agents to express emotion that observers can empathize with, they need to generate gaze- including eye, head, and torso movement- to arbitrary targets, while displaying arbitrary emotional states. Our previous work [18] describes the Gaze Warping Transformation, a method of generating emotionally expressive head and torso movement during gaze shifts that is derived from human movement data. Through an evaluation, it was shown that applying different transformations to the same gaze shift could modify the affective state perceived when the transformed gaze shift was viewed by a human