A robotic chauffeur should reason about spatial information with a variety of scales, dimensions, and ontologies. Rich representations of both the quantitative and qualitative characteristics of space not only enable robust navigation behavior, but also permit natural communication with a human passenger. We apply a hierarchical framework of spatial knowledge inspired by human cognitive abilities, the Hybrid Spatial Semantic Hierarchy, to common navigation tasks: safe motion, localization, map-building, and route planning. We also discuss the straightforward mapping between the variety of ways in which people communicate with a chauffeur and the framework’s heterogeneous concepts of spatial knowledge. We present pilot experiments with a virtual chauffeur.

We present the GIVE-2 Corpus, a new corpus of human instruction giving. The corpus was collected by asking one person in each pair of subjects to guide the other person towards completing a task in a virtual 3D environment with typed instructions. This is the same setting as that of the recent GIVE Challenge, and thus the corpus can serve as a source of data and as a point of comparison for NLG systems that participate in the GIVE Challenge. The instruction-giving data we collect is multilingual (45 German and 63 English dialogues), and can easily be extended to further languages by using our software, which we have made available. We analyze the corpus to study the effects of learning by repeated participation in the task and the effects of the participants ’ spatial navigation abilities. Finally, we present a novel annotation scheme for situated referring expressions and compare the referring expressions in the German and English data. 1.

Abstract—Speaking using unconstrained natural language is an intuitive and flexible way for humans to interact with robots. Understanding this kind of linguistic input is challenging because diverse words and phrases must be mapped into structures that the robot can understand, and elements in those structures must be grounded in an uncertain environment. We present a system that follows natural language directions by extracting a sequence of spatial description clauses from the linguistic input and then infers the most probable path through the environment given only information about the environmental geometry and detected visible objects. We use a probabilistic graphical model that factors into three key components. The first component grounds landmark phrases such as “the computers ” in the perceptual frame of the robot by exploiting co-occurrence statistics from a database of tagged images such as Flickr. Second, a spatial reasoning component judges how well spatial relations such as “past the computers ” describe a path. Finally, verb phrases such as “turn right ” are modeled according to the amount of change in orientation in the path. Our system follows 60 % of the directions in our corpus to within 15 meters of the true destination, significantly outperforming other approaches. I.

Abstract not found

Abstract—Mobile robots that interact with humans in an intuitive way must be able to follow directions provided by humans in unconstrained natural language. In this work we investigate how statistical machine translation techniques can be used to bridge the gap between natural language route instructions and a map of an environment built by a robot. Our approach uses training data to learn to translate from natural language instructions to an automatically-labeled map. The complexity of the translation process is controlled by taking advantage of physical constraints imposed by the map. As a result, our technique can efficiently handle uncertainty in both map labeling and parsing. Our experiments demonstrate the promising capabilities achieved by our approach. Index Terms—Human-robot interaction; instruction following; navigation; statistical machine translation; natural language I.

In this paper, we address the task of mapping high-level instructions to sequences of commands in an external environment. Processing these instructions is challenging—they posit goals to be achieved without specifying the steps required to complete them. We describe a method that fills in missing information using an automatically derived environment model that encodes states, transitions, and commands that cause these transitions to happen. We present an efficient approximate approach for learning this environment model as part of a policygradient reinforcement learning algorithm for text interpretation. This design enables learning for mapping high-level instructions, which previous statistical methods cannot handle. 1 1

Abstract not found

Abstract not found

In this paper, we tackle the challenging problem of guiding pedestrians in buildings. We propose a conceptual model for indoor environments, based only on regions and their boundaries. It needs to be computed just once. Our approach covers different phenomena, in particular irregular, nonconvex regions which are not trivial. Visibility is modelled implicitly and can be determined efficiently. We illustrate by examples how route descriptions can be derived from the model. 1

To my parents, Paul and B.J., for encouraging both wonder and accomplishment. To my wife, Sarah, for her unflagging love, support, and understanding. To all my friends, who have helped in innumerable ways.