By now it is widely accepted that learning a task from scratch, i.e., without any prior knowledge, is a daunting undertaking. Humans, however, rarely attempt to learn from scratch. They extract initial biases as well as strategies how to approach a learning problem from instructions and/or demonstrations of other humans. For learning control, this paper investigates how learning from demonstration can be applied in the context of reinforcement learning. We consider priming the Q-function, the value function, the policy, and the model of the task dynamics as possible areas where demonstrations can speed up learning. In general nonlinear learning problems, only model-based reinforcement learning shows significant speed-up after a demonstration, while in the special case of linear quadratic regulator (LQR) problems, all methods profit from the demonstration. In an implementation of pole balancing on a complex anthropomorphic robot arm, we demonstrate that, when facing the complexities of real signal processing, model-based reinforcement learning offers the most robustness for LQR problems. Using the suggested methods, the robot learns pole balancing in just a single trial after a 30 second long demonstration of the human instructor. 1.

Abstract. This paper proposes cognitive developmental robotics as a new principle for the design of humanoid robots. This principle may provide ways of understanding human beings that go beyond the current level of explanation found in the natural and social sciences. Furthermore, a methodological emphasis on humanoid robots in the design of artificial creatures holds promise because they have many degrees of freedom and sense modalities and, thus, must face the challenges of scalability that are often side stepped in simpler domains. We examine the potential of this new principle as well as issues that are likely to be important to CDR in the future. 1

Many motor skills in humanoid robotics can be learned using parametrized motor primitives as done in imitation learning. However, most interesting motor learning problems are high-dimensional reinforcement learning problems often beyond the reach of current methods. In this paper, we extend previous work on policy learning from the immediate reward case to episodic reinforcement learning. We show that this results in a general, common framework also connected to policy gradient methods and yielding a novel algorithm for policy learning that is particularly well-suited for dynamic motor primitives. The resulting algorithm is an EM-inspired algorithm applicable to complex motor learning tasks. We compare this algorithm to several well-known parametrized policy search methods and show that it outperforms them. We apply it in the context of motor learning and show that it can learn a complex Ball-in-a-Cup task using a real Barrett WAM TM robot arm. 1

This paper explores a newly developing direction of machine learning called ''socially embedded learning". In this research we have been building an office-conversant mobile robot which autonomously moves around in an office environment, actively gathers information through close interaction with this environment including sensing multi-modal data and making dialog with people in the office, and acquires knowledge about the environment with which it ultimately becomes conversant. Here our major concerns are in how the close interaction between the learning system and its social environment can help or accelerate the systems learning process, and what kinds of prepared mechanisms are necessary for the emergence of such interactions. The office-conversant robot is a platform on which we implement our ideas and test their feasibility in a real-world setting. An overview of the system is given and two examples of implemented ideas, i.e. dialog-based map acquisition and route acquisition by following, are described in detail.

A tennis serve and upswing learning robot based on bi-directional theory

Abstract not found

This paper explores a newly developing direction of machine learning called "socially embedded learning". In this research we have been building an office-conversant mobile robot which autonomously moves around in an office environment, actively gathers information through close interaction with this environment including sensing multi-modal data and making dialog with people in the office, and acquires knowledge about the environment with which it ultimately becomes conversant. Here our major concerns are in how the close interaction between the learning system and its social environment can help or accelerate the system's learning process, and what kinds of prepared mechanisms are necessary for the emergence of such interactions. The office-conversant robot is a platform on which we implement our ideas and test their feasibility in a real-world setting. An overview of the system is given and two examples of implemented ideas, i.e. dialog-based map acquisition and route acquisition by...

Abstract- Reinforcement Learning is an essential ability for robots to learn new motor skills. Nevertheless, few methods scale into the domain of anthropomorphic robotics. In order to improve in terms of efficiency, the problem is reduced onto reward-weighted imitation. By doing so, we are able to generate a framework for policy learning which both unifies previous reinforcement learning approaches and allows the derivation of novel algorithms. We show our two most relevant applications both for motor primitive learning (e.g., a complex Ball-in-a-Cup task using a real Barrett WAMTM robot arm) and learning task-space control. I.

Abstract—Reinforcement Learning (RL) methods are widely used for dynamic control tasks. In many cases, these are high risk tasks where the trial and error process may select actions which execution from unsafe states can be catastrophic. In addition, many of these tasks have continuous state and action spaces, making the learning problem harder and unapproachable with conventional RL algorithms. So, when the agent begins to interact with a risky and large state-action spaces task, an important question arises: how can we avoid that the exploration of the state-action space cause damages in the learning (or other) systems. In this paper, we define the concept of risk and address the problem of safe exploration in the context of RL. Our notion of safety is concerned with states that can lead to damage. Moreover, we introduce an algorithm that safely improve suboptimal but robust behaviors for continuous state and action control tasks, and that learns eficiently from the experience gathered from the environment. We report experimental results using the helicopter hovering task from the RL Competition. I.