Abstract. The communication of emotion plays an important role in social interaction. Research in affective display both in the social sciences and in social human-robot interaction has focused almost exclusively on the modalities of vision and audition; however, touch has received disproportionate attention. This paper presents an overview of the Haptic Creature project, where we seek to develop a deeper understanding of affect display through touch in the context of social interaction between human and robot. We also hope to gain knowledge on the role affective touch plays in supporting companionship. Drawing from studies on human-animal interaction, we are developing the Haptic Creature, a robot that mimics a small pet that interacts through touch. Details of the robot and related user studies are presented. 1

Abstract—Joint attention – the idea that humans make inferences from observable behaviors of other humans by attending to the objects and events that these others humans attend to – has been recognized as a critical component in human-robot interactions. While various HRI studies showed that having robots to behave in ways that support human recognition of joint attention leads to better behavioral outcomes on the human side, there are no studies that investigate the detailed time course of interactive joint attention processes. In this paper, we present the results from an HRI study that investigates the exact time course of human multi-modal attentional processes during an HRI word learning task in an unprecedented way. Using novel data analysis techniques, we are able to demonstrate that the temporal details of human attentional behavior are critical for understanding human expectations of joint attention in HRI and that failing to do so can force humans into assuming unnatural behaviors. Keywords-human-robot interaction; joint attention I.

In this paper, we present an integrated planning and robotic architecture that actively directs an agent engaged in an urban search and rescue (USAR) scenario. We describe three salient features that comprise the planning component of this system, namely (1) the ability to plan in a world open with respect to objects, (2) execution monitoring and replanning abilities, and (3) handling soft goals, and detail the interaction of these parts in representing and solving the USAR scenario at hand. We show that though insufficient in an individual capacity, the integration of this trio of features is sufficient to solve the scenario that we present. We test our system with an example problem that involves soft and hard goals, as well as goal deadlines and action costs, and show via an included video that the planner is capable of incorporating sensing actions and execution monitoring in order to produce goal-fulfilling plans that maximize the net benefit accrued.

Abstract — Autonomous robots that operate in real-world domains face multiple challenges that make planning and goal selection difficult. Not only must planning and execution occur in real time, newly acquired knowledge can invalidate previous plans, and goals and their utilities can change during plan execution. However, these events can also provide opportunities, if the architecture is designed to react appropriately. We present here an architecture that integrates the SapaReplan planner with the DIARC robot architecture, allowing the architecture to react dynamically to changes in the robot’s goal structures. I.

Complex robots that interact naturally with humans require the integration, coordination and maintenance of many diverse software components and algorithms. An architecture that incorporates explicit knowledge about the relationships among these components and the overall system state can be used for introspection and consequently to reason about the best configurations of the computing environment under changing conditions; potential uses include maintaining the system’s integrity, promoting its health, and providing the ability to dynamically reconfigure system components (e.g., after component failure). In this paper, we describe a rudimentary reasoning system, part of our Distributed Integrated Affect Reflection Cognition (DIARC) architecture for human-robot interaction, that can autonomously perform failure detection, failure recovery, and system reconfiguration of distributed architectural components to ensure sustained operation and interactions. We demonstrate the functionality and utility of the proposed mechanisms on a robot, where architectural components are forcefully removed by hand and automatically recovered by the system while the robot is continuing its interactions with humans as part of a joint human-robot task.

Multitasking has become an integral part of work environments, even though people are not well-equipped cognitively to handle numerous concurrent tasks effectively. Systems that support such multitasking may produce better performance and less frustration. However, without understanding the user’s internal processes, it is difficult to determine optimal strategies for adapting interfaces, since all multitasking activity is not identical. We describe two experiments leading toward a system that detects cognitive multitasking processes and uses this information as input to an adaptive interface. Using functional near-infrared spectroscopy sensors, we differentiate four cognitive multitasking processes. These states cannot readily be distinguished using behavioral measures such as response time, accuracy, keystrokes or screen contents. We then present our human-robot system as a proof-of-concept that uses real-time cognitive state information as input and adapts in response. This prototype system serves as a platform to study interfaces that enable better task switching, interruption management, and multitasking. Author Keywords fNIRS, near-infrared spectroscopy, multitasking, interruption,

Mental modeling is crucial for natural humanrobot interactions (HRI). Yet, effective mechanisms that enable reasoning about and communication of mental states are not available. We propose to utilize adverbial cues, routinely employed by humans, for this goal and present a novel algorithm that integrates adverbial modifiers with belief revision and expression, phrasing utterances based on Gricean conversational maxims. The algorithm is demonstrated in a simple HRI scenario. 1

Efficient collaborations between interacting agents, be they humans, virtual or embodied agents, require mutual recognition of the goal, appropriate sequencing and coordination of each agent's behavior with others, and making predictions from and about the likely behavior of others. Moment-by-moment eye gaze plays an important role in such interaction and collaboration. In light of this, we used a novel experimental paradigm to systematically investigate gaze patterns in both human-human and human-agent interactions. Participants in the study were asked to interact with either another human or an embodied agent in a joint attention task. Fine-grained multimodal behavioral data were recorded including eye movement data, speech, first-person view video, which were then analyzed to discover various behavioral patterns. Those patterns show that human participants are highly sensitive to momentary multimodal behaviors generated by the social partner (either another human or an artificial agent) and they rapidly adapt their gaze behaviors accordingly. Our results from this data-driven approach provide new findings for understanding micro-behaviors in human-human communication which will be critical for the design of artificial agents that can generate human-like gaze behaviors and engage in multimodal interactions with