This dissertation would not have been possible without the help and support of many people. Most of all, I would like to extend my gratitude to Rod Grupen for many years of inspiring work, our discussions, and his guidance. Without his support and vision, I cannot imagine that the journey would have been as enormously enjoyable and rewarding as it turned out to be. I am very excited about what we discovered during my time at UMass, but there is much more to be done. I look forward to what comes next! In addition to providing professional inspiration, Rod was a great person to work with and for—creating a warm and encouraging laboratory atmosphere, motivating us to stay in shape for his annual half-marathons, and ensuring a sufficient amount of cake at the weekly lab meetings. Thanks for all your support, Rod! I am very grateful to my thesis committee—Andy Barto, David Jensen, and Rachel Keen—for many encouraging and inspirational discussions. Their comments and feedback significantly contributed to the form of this document. I would especially

Abstract — We present an initial demonstration of dexterous mobility using the uBot-5, a dynamically balancing mobile manipulator. Dexterous mobility refers generally to a level of bodily resourcefulness that permits the autonomous reassignment of effectors for the purpose of maintaining mobility in a variety of situations. We begin by describing a set of postural stability controllers in terms of a small number of simple control objectives. We then show how the resulting postures support dexterous mobility by enabling a new “knuckle walking ” mobility mode. In a preliminary experiment, we develop this mobility mode by formulating a practical reinforcement learning problem that allows the robot to learn an efficient gait on-line in a single trial. I.

Abstract—One important role of an agent’s motivational system is to choose, at any given moment, which of a number of skills the agent should attempt to improve. Many researchers have suggested “intrinsically motivated ” systems that receive internal reward for model learning progress, but for the most part this notion has not been applied with respect to skill competence, or to choose between skills. In this paper we propose an agent motivated to gain competence in its environment by learning a number of skills, addressing head-on the mechanism of competence progress motivation for the purpose of governing the efficient learning of skills. We demonstrate this new approach in a simple illustrative domain and show that it outperforms a naïve agent, achieving higher competence faster by focusing attention and learning effort on skills for which progress can be made while ignoring those skills that are already learned or are at the moment too difficult. I.

Personal robotics is an area in which robot behavior is in service to few (or single) clients. This paper argues that the problems of human detection and recognition can be approached with simple yet efficient techniques that provide useful information to personal robots. By combining and taking advantage of coarse information such as motion, activities, shape, and color attributes, simple probabilistic inference algorithms can be applied to help a robot to become aware of nearby humans and their identities. Experimental results show that these simple models can be used to detect human presence robustly against a naturally clutterd and ambiguous background and perform well in a recognition test consisting of 10 subjects. Since this approach does not rely on the faces as crucial cue for detection or recognition, it can function under situations where conventional techniques would fail. Moreover, the simple model offers dramatic improvement in computation efficiency and can be used for robots to engage real-time interaction with human. Keywords human detection, human recognition, human centric robotics, learning 1.

Abstract — Assisting humans in their daily lives requires robots to be proficient in manual tasks and effective in communicating states/intentions with human users. This paper advocates a learning approach for the development of communicative behavior in robots and favors a uniform means of learning communicative actions and manual skills in the same framework. In fact, this work argues for a critical relationship between the structure of motor skills and the structure required to communicate effectively. We show how to reuse manual behavior for conveying intentions to humans and to do so in the same grounded manner as the robot learns to interact with other objects in the environment. The learning framework and preliminary human-robot interaction experiments are presented, where a humanoid robot incrementally builds and refines communicative actions by discovering the utility of manipulation behavior in the presence of humans. The learning results from 18 subjects provide support for the hypothesized benefits of our approach that behavior reuse made learning from relatively few interactions possible and the robust manual behavioral basis kept the subjects interested. The approach presented in this paper compliments other efforts in the field as it grounds social behaviors, allowing them to be more adaptive to context changes or variations in human user preferences. I.

Abstract — Software tools for programming autonomous systems that are embedded in unstructured environments are increasingly important in robotics. We introduce a layered software architecture designed to facilitate the construction of hierarchical models for adaptive control programs that are learned and that can be transferred to related contexts and new robots. We focus on the interface between a robot’s sensory and motor resources and processes that learn autonomously by exploring effects of the robot’s actions. We provide an implementation of this interface called the Control Basis Application Programming Interface (CBAPI) that is designed to create hierarchical behavior and implicit knowledge out of closed-loop control primitives. The CBAPI provides a natural combinatorial means of building closed-loop controllers by combining sensory and motor resources. By so doing, it supports a variety of techniques for structuring stochastic exploration and interactive machine learning. Moreover, it provides for a natural implicit knowledge representation. We believe that the CBAPI represents a programming interface for adaptive control programs that advances the state-of-the-art in robotic software environments. I.

LEARNING techniques are increasingly being used in today’s complex robotic systems. Robots are expected to deal with a large variety of tasks using their high-dimensional and complex bodies, to manipulate objects and also, to interact with humans in an intuitive and friendly way. In this new setting, not all relevant information is available at design time, and robots should typically be able to learn, through self-experimentation or through human–robot interaction, how to tune their innate perceptual-motor skills or to learn, cumulatively, novel skills that were not preprogrammed initially. In a word, robots need to have the capacity to develop in an open-ended manner and in an open-ended environment, in a way that is analogous to human development which combines genetic and epigenetic factors. This challenge is at the center of the developmental robotics field [7], [35]–[37]. Among the various

Abstract not found

This paper presents a learning framework that enables a robot to learn comprehensive policies autonomously from a series of incrementally more challenging tasks designed by a human teacher. Psychologists have shown that human infants rapidly acquire general strategies and then extend that behavior with contingencies for new situations. This strategy allows an infant to quickly acquire new behavior and then to refine it over time. The psychology literature calls such compensatory action prospective behavior and it has been identified as an important problem in robotics as well. In this paper, we provide an algorithm for learning prospective behavior to accommodate special-purpose situations that can occur when a general-purpose schema is applied to challenging new cases. The algorithm permits a robot to address complex tasks incrementally while reusing existing behavior as much as possible. First, we motivate prospective behavior in human infants and in common robotic tasks. We introduce an algorithm that searches for places in a schema where compensatory actions can effectively avoid predictable future errors. The algorithm is evaluated on a simple grid-world navigation problem. Results show that learning performance improves significantly over an equivalent flat learning formulation by re-using knowledge as appropriate and extending behavior only when necessary. We conclude with a discussion of where prospective repair of general-purpose behavior can play important roles in the development of behavior for effective human-robot interaction.

This paper presents an intrinsic motivation function called the multi-modal imperative (MMI) that can be used by sensorimotor systems to learn deep control knowledge about behavioral affordances [1]. It builds upon the control basis