We present an approach to teach incrementally human gestures to a humanoid robot. The learning process consists of first projecting the movement data in a latent space and encoding the resulting signals in a Gaussian Mixture Model (GMM). We compare the performance of two incremental training procedures against a batch training procedure. Qualitative and quantitative evaluations are performed on data acquired from motion sensors attached to a human demonstrator and data acquired by kinesthetically demonstrating the task to the robot. We present experiments to show that these different modalities can be used to teach incrementally basketball officials ’ signals to a HOAP-3 humanoid robot. 1.

Abstract — Ubiquitous robots need the ability to adapt their behaviour to the changing situations and demands they will encounter during their lifetimes. In particular, non-technical users must be able to modify a robot’s behaviour to enable it to perform new, previously unknown tasks. Learning from Demonstration is a viable means to transfer a desired control policy onto a robot and Mixed-Initiative Control provides a method for smooth transitioning between learning and acting. We present a learning system (Dogged Learning) that combines Learning from Demonstration and Mixed Initiative Control to enable lifelong learning for unknown tasks. We have implemented Dogged Learning on a Sony Aibo and successfully taught it behaviours such as mimicry and ball seeking. I.

Abstract — We are interested in transferring control policies for arbitrary tasks from a human to a robot. Using interactive demonstration via teleoperation as our transfer scenario, we cast learning as statistical regression over sensor-actuator data pairs. Our desire for interactive learning necessitates algorithms that are incremental and realtime. We examine Locally Weighted Projection Regression, a popular robotic learning algorithm, and Sparse Online Gaussian Processes in this domain on one synthetic and several robot-generated data sets. We evaluate each algorithm in terms of function approximation, learned task performance, and scalability to large data sets. I. INTRODUCTION AND RELATED WORK In this paper we address the problem of Policy transfer, how a control policy (π) for some unknown task, latent in the mind of a human, can be transitioned onto a robot. The

Robot programming by demonstration (RPD) covers methods by which a robot learns new skills through human guidance. We present an interactive, multimodal RPD framework using active teaching methods that places the human teacher in the robot’s learning loop. Two experiments are presented in which observational learning is first used to demonstrate a manipulation skill to a HOAP-3 humanoid robot by using motion sensors attached to the teacher’s body. Then, putting the robot through the motion, the teacher incrementally refines the robot’s skill by moving its arms manually, providing the appropriate scaffolds to reproduce the action. An incremental teaching scenario is proposed based on insights from various fields addressing developmental, psychological, and social issues related to teaching mechanisms in humans. Based on this analysis, different benchmarks are suggested to evaluate the setup further. In a robot programming by demonstration (RPD) framework, a robot learns new skills through the help of a human instructor (Billard & Siegwart, 2004). Traditionally, RPD tends to consider the human user as an expert model who performs a task while the robot observes passively the demonstration (Ikeuchi & Suchiro, 1992; Kuniyoshi, Inaba, & Inoue, 1994). However, in humans, teaching is a social and bidirectional process in which teacher and learner are both active. Instead of considering the teacher solely as a model of successful expert behavior, recent work has referred to the teacher-learner couple as a We gratefully acknowledge Chrystopher L. Nehaniv, Aris Alissandrakis, Joe Saunders, Nuno Otero and Kerstin Dautenhahn for the useful exchanges of thoughts concerning social cues and feedback as well as the user experience and evaluation issues tackled by the Cogniron project. We would also like to acknowledge the four anonymous reviewers for their very useful comments on an earlier version of the manuscript.

We present a neural network approach to early motor learning. The goal is to explore the needs for bootstrapping the control of hand movements in a biologically plausible learning scenario. The model is applied to the control of hand postures of the humanoid robot ASIMO by means of full upper body movements. For training, we use an efficient online scheme for recurrent reservoir networks consisting of supervised backpropagation-decorrelation output adaptation and an unsupervised intrinsic plasticity reservoir optimization. We demonstrate that the network can acquire accurate inverse models for the highly redundant ASIMO, applying bi-manual target motions and exploiting all upper body degrees of freedom. We show that very few, but highly symmetric training data is sufficient to generate excellent generalization capabilities to untrained target motions. We also succeed in reproducing real motion recorded from a human demonstrator, massively differing from the training data in range and dynamics. The demonstrated generalization capabilities provide a fundamental prerequisite for an autonomous and incremental motor learning in an developmentally plausible way. Our exploration process – though not yet fully autonomous – clearly shows that goal- directed exploration can, in contrast to “babbling” of joints angles, be done very efficiently even for many degrees of freedom and non-linear kinematic configurations as ASIMOs.

Abstract — This paper considers prediction of slip from a distance for wheeled ground robots using visual information as input. Large amounts of slippage which can occur on certain surfaces, such as sandy slopes, will negatively affect rover mobility. Therefore, obtaining information about slip before entering a particular terrain can be very useful for better planning and avoiding terrains with large slip. The proposed method is based on learning from experience and consists of terrain type recognition and nonlinear regression modeling. After learning, slip prediction is done remotely using only the visual information as input. The method has been implemented and tested offline on several off-road terrains including: soil, sand, gravel, and woodchips. The slip prediction error is about 20 % of the step size. I.

Abstract — We seek to enable users to teach personal robots arbitrary tasks so that the robot can better perform as the user desires without explicit programming. Robot learning from demonstration is an approach well-suited to this paradigm, as a robot learns new tasks from observations of the task itself. Many current robot learning algorithms require the existence of basic behaviors that can be combined to perform the desired task. However, robots that exist in the world for long timeframes and learn many tasks over their lifetime may exhaust this basis set and need to move beyond it. In particular, we are interested in a robot that must learn to perform an unknown task for which its built in behaviors may not be appropriate. We demonstrate a learning paradigm that is capable of learning both low-level motion primitives (locomotion and manipulation) and high-level tasks built on top of them from interactive demonstration. We apply nonparametric regression within this framework towards learning a complete robot soccer player and successfully teach a robot dog to first walk, and then to seek and acquire a ball. I.

Abstract—We present a method for learning potential-based policies from constrained motion data. In contrast to previous approaches to direct policy learning, our method can combine observations from a variety of contexts where different constraints are in force, to learn the underlying unconstrained policy in form of its potential function. This allows us to generalise and predict behaviour where novel constraints apply. As a key ingredient, we first create multiple simple local models of the potential, and align those using an efficient algorithm. We can then detect and discard unsuitable subsets of the data and learn a global model from a cleanly pre-processed training set. We demonstrate our approach on systems of varying complexity, including kinematic data from the ASIMO humanoid robot with 22 degrees of freedom. I.

Abstract. We present a review of recent trends in cognitive robotics that deal with online learning approaches to the acquisition of knowledge, control strategies and behaviors of a cognitive robot or agent. Along this line we focus on the topics of object recognition in cognitive vision, trajectory learning and adaptive control of multi-DOF robots, task learning from demonstration, and general developmental approaches in robotics. We argue for the relevance of online learning as a key ability for future intelligent robotic systems to allow flexible and adaptive behavior within a changing and unpredictable environment. 1

Abstract. Many everyday human skills can be framed in terms of performing some task subject to constraints imposed by the task or the environment. Constraints are usually not observable and frequently change between contexts. In this chapter, we explore the problem of learning control policies from data containing variable, dynamic and non-linear constraints on motion. We discuss how an effective approach for doing this is to learn the unconstrained policy in awaythatisconsistent with the constraints. Wethengoontodiscussseveral recent algorithms for extracting policies from movement data, where observations are recorded under variable, unknown constraints. We review a number of experiments testing the performance of these algorithms and demonstrating how the resultant policy models generalise over constraints allowing prediction of behaviour under unseen settings where new constraints apply. 1