We present a comprehensive survey of robot Learning from Demonstration (LfD), a technique that develops policies from example state to action mappings. We introduce the LfD design choices in terms of demonstrator, problem space, policy derivation and performance, and contribute the foundations for a structure in which to categorize LfD research. Specifically, we analyze and categorize the multiple ways in which examples are gathered, ranging from teleoperation to imitation, as well as the various techniques for policy derivation, including matching functions, dynamics models and plans. To conclude we discuss LfD limitations and related promising areas for future research.

We present an approach to teach incrementally human gestures to a humanoid robot. The learning process consists of first projecting the movement data in a latent space and encoding the resulting signals in a Gaussian Mixture Model (GMM). We compare the performance of two incremental training procedures against a batch training procedure. Qualitative and quantitative evaluations are performed on data acquired from motion sensors attached to a human demonstrator and data acquired by kinesthetically demonstrating the task to the robot. We present experiments to show that these different modalities can be used to teach incrementally basketball officials ’ signals to a HOAP-3 humanoid robot. 1.

The identification of any form of social learning, imitation, copying or mimicry presupposes a notion of correspondence between two autonomous agents. Judging whether a behavior has been transmitted socially requires the observer to identify a mapping between the demonstrator and the imitator. If the demonstrator and imitator have similar bodies, e.g. are animals of the same species, of similar age, and of the same gender, then to a human observer an obvious correspondence is to map the corresponding body parts: left arm of demonstrator maps to left arm of imitator, right eye of demonstrator maps to right eye of imitator, tail of demonstrator maps to tail of imitator. There is also an obvious correspondence of actions: raising the left arm by the model corresponds to raising the left arm by the imitator, production of vocal signals by the model corresponds to the production of acoustically similar ones by the imitator, picking up a fruit by the demonstrator corresponds to picking up a fruit of the same type by the imitator. Furthermore, there is a correspondence in sensory experience: audible sounds, a touch, visible objects and colors, and so on evidently seem to be detected and experienced in similar ways. What to take as the correspondence seems relatively clear in this case. As humans, we are good at imitating and at recognizing such correspondences. It is also clear that most other animals, robots, and software programs may in fact generally fail to recognize any such correspondences. To judge a produced behavior to be a copy of an observed one, we require at least that it respects some such correspondence. The faithfulness or precision of the behavioral match can obviously vary, and no absolute cutoff or threshold exists defining success as opposed to failure of behavioral matching. But one can study the degree of success using various metrics and measures of correspondence (Nehaniv & Dautenhahn, 2001; also see below). Moreover, it turns out that the “obvious” correspondences between similar bodies mentioned above are not the only ones possible. Consider a human imitating another one that is facing her: if the demonstrator raises her left arm, should the imitator raise her own left arm? Or should she raise her right, to make a "mirror image" of the demonstrator's actions? If the demonstrator picks up a brush, should an imitator pick up the same brush? Or just another brush of the same type? If the demonstrator opens a container to get at chocolate inside, should the imitator open a similar container in the same way – e.g. by unwrapping but not tearing the surrounding paper?, or is it enough just to open the container somehow? The different possible answers to these questions presuppose different correspondences. If a child watches a teacher solving subtraction problems in arithmetic, and then solves for the first time similar but not identical problems on its own, social learning has occurred. But what type of correspondence is at work here? In China and Japan, the ideographic character for “to imitate” also means “to learn” or “to study”. By going through the motions of an algorithm for solving sample problems, students everywhere are able to learn how to solve similar ones, of course without necessarily gaining understanding of why the procedures they have learned work. In this chapter, for lack of a better term, we shall use the word “imitator” to refer to any autonomous agent performing a candidate behavioral match. The use of this word here does not entail any particular mechanism of matching or any particular type of social learning. In what follows, we shall describe how different matching phenomena arise depending on the criteria employed in generating the behavior of the imitator. For example, goal emulation, stimulus enhancement, mimicry, and so on, will all be cast as solutions to correspondence problems with different particular selection criteria.

Introduction This chapter presents the agent-based perspective on imitation. In this perspective, imitation is best considered as the behavior of an autonomous agent in relation to its environment, including other autonomous agents. We argue that such a perspective helps unfold the full potential of research on imitation and helps in identifying challenging and important research issues. We first explain the agent-based perspective and then discuss it in the context of particular research issues in studies with animals and artifacts, with reference to chapters presented in this book. At the end of the chapter we briefly introduce the individual contributions to this book and provide a roadmap that helps the reader in navigating through the exciting and highly interwoven themes that are presented in this book. In order to focus discussions, we explain the agent-based perspective with particular consideration of the correspondence

Abstract not found

Abstract — We present a strategy whereby a robot acquires the capability to learn by imitation following a developmental pathway consisting on three levels: (i) sensory-motor coordination, (ii) world interaction, (iii) imitation. With these stages, the system is able to learn tasks by imitating human demonstrators. We describe results of the different developmental stages, involving perceptual and motor skills, implemented in our humanoid robot, Baltazar. At each stage, the system’s attention is drawn towards different entities: its own body and later on, objects and people. Our main contributions are the general architecture and the implementation of all the necessary modules until imitation capabilities are eventually acquired by the robot. Also several other contributions are made at each level: learning of sensory-motor maps for redundant robots, a novel method for learning how to grasp objects and a framework for learning task description from observation for program-level imitation. Finally, vision is used extensively as the sole sensing modality (sometimes in a simplified setting) avoiding the need for special data-acquisition hardware. Index Terms — Humanoid Robots, development, imitation I.

This study presents experiments on the learning of object handling behaviors by a small humanoid robot using a dynamic neural network model, the recurrent neural network with parametric bias (RNNPB). The first experiment showed that after the robot learned different types of ball handling behaviors using human direct teaching, the robot was able to generate adequate ball handling motor sequences situated to the relative position between the robot’s hands and the ball. The same scheme was applied to a block handling learning task where it was shown that the robot can switch among learned different block handling sequences, situated to the ways of interaction by human supporters. Our analysis showed that entrainment of the internal memory structures of the RNNPB through the interactions of the objects and the human supporters are the essential mechanisms for those observed situated behaviors of the robot.

. Interactive behavior of biological agents represents an important area in life as we know it. Behavior matching and imitation may serve as fundamental mechanisms for the development of societies and individuals. Imitation and observational learning as means for acquiring new behaviors also represent a largely untapped resource for robotics and artificial life --- both in the study of life as it could be and for applications of biological tricks to synthetic worlds. This paper describes a new general imitating mechanism called ALICE (Action Learning for Imitation via Correspondences between Embodiments) that addresses the important correspondence problem in imitation. The mechanism is implemented and illustrated on the chessworld test-bed that was used in previous work to address the effects of agent embodiment, metrics and granularity when learning how to imitate another. The performance of the imitating agent is shown to improve when ALICE is complementing its imitation behavior generating mechanism. 1.

This chapter reviews several approaches to the problem of learning by imitation in robotics. We start by describing several cognitive processes identified in the literature as necessary for imitation. We then proceed by surveying different approaches to this problem, placing particular emphasys on methods whereby an agent first learns about its own body dynamics by means of self-exploration and then uses this knowledge about its own body to recognize the actions being performed by other agents. This general approach is related to the motor theory of perception, particularly to the mirror neurons found in primates. We distinguish three fundamental classes of methods, corresponding to three abstraction levels at which imitation can be addressed. As such, the methods surveyed herein exhibit behaviors that range from raw sensory-motor trajectory matching to high-level abstract task replication. We also discuss the impact that knowledge about the world and/or the demonstrator can have on the particular behaviors exhibited.

Imitation represents a useful and promising alternative to programming robots. The approach presented here is based on two functional elements used by humans to understand and perform actions. These elements are: the body schema and the body percept. The first one is a representation of the body containing information of the body's capabilities. The body percept is a snapshot of the body and its relation with the environment at a given instant. These elements are believed to interact between each other generating among other abilities, the ability to imitate. This paper presents our approach to robot imitation and experimental results, where a robot is able to imitate the movements of a human demonstrator via its visual observations.