Abstract — Robots that work alongside us in our homes and workplaces could extend the time an elderly person can live at home, provide physical assistance to a worker on an assembly line, or help with household chores. In order to assist us in these ways, robots will need to successfully perform manipulation tasks within human environments. Human environments present special challenges for robot manipulation since they are complex, dynamic, uncontrolled, and difficult to perceive reliably. In this paper we present a behavior-based control system that enables a humanoid robot, Domo, to help a person place objects on a shelf. Domo is able to physically locate the shelf, socially cue a person to hand it an object, grasp the object that has been handed to it, transfer the object to the hand that is closest to the shelf, and place the object on the shelf. We use this behavior-based control system to illustrate three themes that characterize our approach to manipulation in human environments. The first theme, cooperative manipulation, refers to the advantages that can be gained by having the robot work with a person to cooperatively perform manipulation tasks. The second theme, task relevant features, emphasizes the benefits of carefully selecting the aspects of the world that are to be perceived and acted upon during a manipulation task. The third theme, let the body do the thinking, encompasses several ways in which a robot can use its body to simplify manipulation tasks. 1 Fig. 1. The humanoid robot Domo used in this paper. I.

depends on the availability of adequate experimental platforms. In this paper, we describe an ongoing effort at the University of Massachusetts Amherst to construct a hardware platform with redundant kinematic degrees of freedom, a comprehensive sensor suite, and significant end-effector capabilities for manipulation. In our research, we pursue an end-effector centric view of autonomous mobile manipulation. In support of this view, we are developing a comprehensive software suite to provide a high level of competency in robot control and perception. This software suite is based on a multi-objective, tasklevel motion control framework. We use this control framework to integrate a variety of motion capabilities, including taskbased force or position control of the end-effector, collision-free global motion for the entire mobile manipulator, and mapping and navigation for the mobile base. We also discuss our efforts in developing perception capabilities targeted to problems in autonomous mobile manipulation. Preliminary experiments on our UMass Mobile Manipulator (UMan) are presented. I.

This paper reports about our investigation on action understanding in the brain. We review recent results of the neurophysiology of the mirror system in the monkey. Based on these observations we propose a model of the brain systems responsible for action recognition, in which the link between object affordances and action understanding is explicitly considered. To support our hypothesis we describe two experiments where some aspects of the model have been implemented. In the first experiment an action recognition system is trained by using data recorded from human movements which include kinesthetic, tactile, and visual information. In the second experiment, the model is partially implemented on a humanoid robot which learns to mimic simple actions performed by a human subject on different objects. These experiments show that motor information can have a significant role in interpretation of actions and that a mirror-like representation can be developed autonomously as a result of the interaction between an individual and the environment.

We introduce an approach for physically-grounded natural language interpretation by robots which reacts appropriately to unanticipated physical changes in the environment and dynamically assimilates new information pertinent to ongoing tasks. At the core of the approach is a model of object schemas that enables a robot to encode beliefs about physical objects in its environment using collections of coupled processes responsible for sensorimotor interaction. These interaction processes run concurrently in order to ensure responsiveness to the environment, while coordinating sensorimotor expectations, action planning, and language use. The model has been implemented on a robot that manipulates objects on a tabletop in response to verbal input. The implementation responds to verbal requests such as “Group the green block and the red apple, ” while adapting in real-time to unexpected physical collisions and taking opportunistic advantage of any new information it may receive through perceptual and linguistic channels.

Abstract. One of the first steps of any visual system is that of locating suitable interest points, ‘salient regions’, in the scene, to detect events, and eventually to direct gaze toward these locations. In the last few years, object-based visual attention models have received an increasing interest in computational neuroscience and in computer vision, the problem, in this case, being that of creating a model of ‘objecthood ’ that eventually guides a saliency mechanism. We present here an model of visual attention based on the definition of ‘proto-objects ’ and show its instantiation on a humanoid robot. Moreover we propose a biological plausible way to learn certain Gestalt rules that can lead to proto-objects. 1 Visual Attention Spatial attention is often assimilated to a sort of ‘filter ’ of the incoming information, a ‘spotlight’, an internal eye or a ‘zoom lens’. In particular it is believed to be deployed as a spatial gradient, centered on a particular location. Even if supported by numerous findings (see [1] for a review), this view does not stress

Abstract — Roboticists are working towards the realization of autonomous mobile manipulators that can perform useful tasks in human environments. These environments pose a significant challenge because of their complexity and inherent uncertainty. They are characterized by having a high dimensional state space. Consequently, performing tasks in these unstructured environments remains a challenge. Recently, researchers have been successful in developing skills that can handle the complexity of unstructured environments. We hypothesize that those successes are due to a careful implementation that is able to reduce the complexity of the state space, and render the respective problems tractable. In this paper, we analyze this increasing body of literature, in an attempt to extract the common ideas that enable the reduction of the state space. Based on these commonalities, we propose a set of guidelines to facilitate progress for autonomous mobile manipulation in unstructured environments. I.

Robots and humans receive partial, fragmentary hints about the world’s state through their respective sensors. In this paper, we focus on some fundamental problems in perception that have attracted the attention of researchers in both robotics and infant development: object segregation, intermodal inte-gration, and the role of embodiment. We concentrate on identifying points of contact between the two fields, and also important questions identified in one field and not yet addressed in the other. For object segregation, both fields have examined the idea of using “key events ” where perception is in some way simplified and the infant or robot acquires knowledge that can be exploited at other times. We examine this parallel research in some detail. We propose that the identification of the key events themselves constitutes a point of contact between the fields. And although the specific algorithms used in robots are not easy to relate to infant development, the overall “algorithmic skeleton ” formed by the set of algorithms needed to identify and exploit key events may in fact form a basis for mutual dialogue.

Abstract — Learning object representations by exploration is of great importance for cognitive robots that need to learn about their environment without external help. In this paper we present sensorimotor processes that enable the robot to observe grasped objects from all relevant viewpoints, which makes it possible to learn viewpoint independent object representations. Taking control of the object allows the robot to focus on relevant parts of the images, thus bypassing potential pitfalls of pure bottom-up attention and segmentation. We propose a systematic method to control a robot in order to achieve a maximum range of motion across the 3-D view sphere. This is done by exploiting the task redundancies typically found on a humanoid arm and by avoiding joint limits of the robot. The proposed method brings the robot into configurations that are appropriate for observing objects. It enables us to acquire a wider range of snapshots without regrasping the object. I.

Abstract — Interactive perception augments the process of perception with physical interactions. By adding interactions into the perceptual process, manipulating the environment becomes part of the effort to learn task-relevant information, leading to more reliable task execution. Interactions include obstruction removal, object repositioning, and object manipulation. In this paper, we show how to extract kinematic properties from novel objects. Many objects in human environments, such as doors, drawers, and hand tools, contain inherent kinematic degrees of freedom. Knowledge of these degrees of freedom is required to use the objects in their intended manner. We demonstrate how a simple algorithm enables the construction of kinematic models for such objects, resulting in knowledge necessary for the correct operation of those objects. The simplicity of the framework and its effectiveness, demonstrated in our experimental results, indicate that interactive perception is a promising perceptual paradigm for autonomous mobile manipulation. I.

From exploration to imitation: using learnt internal models to imitate others Abstract. We present an architecture that enables asocial and social learning mechanisms to be combined in a unified framework on a robot. The robot learns two kinds of internal models by interacting with the environment with no a priori knowledge of its own motor system: internal object models are learnt about how its motor system and other objects appear in its sensor data; internal control models are learnt by babbling and represent how the robot controls objects. These asocially-learnt models of the robot’s motor system are used to understand the actions of a human demonstrator on objects that they can both interact with. Knowledge acquired through self-exploration is therefore used as a bootstrapping mechanism to understand others and benefit from their knowledge. 1