Abstract—Navigation is an essential element of many remote robot operations including search and rescue, reconnaissance, and space exploration. Previous reports on using remote mobile robots suggest that navigation is difficult due to poor situation awareness. It has been recommended by experts in human–robot interaction that interfaces between humans and robots provide more spatial information and better situational context in order to improve an operator’s situation awareness. This paper presents an ecological interface paradigm that combines video, map, and robotpose information into a 3-D mixed-reality display. The ecological paradigm is validated in planar worlds by comparing it against the standard interface paradigm in a series of simulated and realworld user studies. Based on the experiment results, observations in the literature, and working hypotheses, we present a series of principles for presenting information to an operator of a remote robot. Index Terms—3-D interface, augmented-virtuality, human– robot interaction, information presentation, teleoperation, USAR-

Creating a human-robot interface is a daunting experience. Capabilities and functionalities of the interface are dependent on the robustness of many different sensor and input modalities. For example, object recognition poses problems for state-of-the-art vision systems. Speech recognition in noisy environments remains problematic for acoustic systems. Natural language understanding and dialog are often limited to specific domains and baffled by ambiguous or novel utterances. Plans based on domain-specific tasks limit the applicability of dialog managers. The types of sensors used limits spatial knowledge and understanding and constrains cognitive issues, such as perspective-taking. In this research, we are integrating several modalities, such as vision, audition, and natural language understanding to leverage the existing strengths of each modality and overcome individual

The XWand is a wireless UI device that enables styles of natural interaction with intelligent environments. The XWand system exploits human intuition, allowing control of everyday objects through pointing and gesturing. We describe the hardware device and then examine several approaches to gesture recognition. We discuss results from experiments using a linear time warping method, a dynamic time warping (DTW) method, and a hidden Markov modelbased method (HMM).

Abstract not found

Contents 1 Introduction and Motivation 1 2 Related Work 2 3 System Design 4 3.2 Determining the Environment Configuration . . . . . . . . . . . . . . . . . . . . . . . . 6 3.3 Computing Subgoals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.4 Associating Subgoals in the Environment . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.5 Mapping Demonstrations to the Same Frame . . . . . . . . . . . . . . . . . . . . . . . 10 3.6 Determining Task Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.7 Performing Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.8 User Modification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.9 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4 Sample Implementation 16 4.2 Determining the Environment Configuration . . . . . . . . . . . . . . . . . . . . . . . . 17 4.3 Computing

In this paper, we describe Dita, a gesture-based control system for a mobile phone user to interact with surrounding devices in a smart home setting. It enables the user to draw simple gestures in free space to select and control a networked device. It devises a high-resolution, acoustic based localization and tracking scheme to trace the gesture-bearing phone’s movement trajectory at runtime. It further uses a simple and robust scheme to recognize and interpret the intended gesture in a context-aware fashion. Our prototype implementation has confirmed that gesture-based humandevice interaction can be built with today’s commodity devices while not relying on sophisticated hardware or elaborated setup. Our evaluation shows that the system is effective and reliable in recognizing simple gestures commonly used in daily life. 1