We consider the problem of grasping novel objects in cluttered environments. If a full 3-d model of the scene were available, one could use the model to estimate the stability and robustness of different grasps (formalized as form/force-closure, etc); in practice, however, a robot facing a novel object will usually be able to perceive only the front (visible) faces of the object. In this paper, we propose an approach to grasping that estimates the stability of different grasps, given only noisy estimates of the shape of visible portions of an object, such as that obtained from a depth sensor. By combining this with a kinematic description of a robot arm and hand, our algorithm is able to compute a specific positioning of the robot’s fingers so as to grasp an object. We test our algorithm on two robots (with very different arms/manipulators, including one with a multi-fingered hand). We report results on the task of grasping objects of significantly different shapes and appearances than ones in the training set, both in highly cluttered and in uncluttered environments. We also apply our algorithm to the problem of unloading items from a dishwasher.

Abstract — We present a novel vision-based grasp point detection algorithm that can reliably detect the corners of a piece of cloth, using only geometric cues that are robust to variation in texture. Furthermore, we demonstrate the effectiveness of our algorithm in the context of folding a towel using a generalpurpose two-armed mobile robotic platform without the use of specialized end-effectors or tools. The robot begins by picking up a randomly dropped towel from a table, goes through a sequence of vision-based re-grasps and manipulations— partially in the air, partially on the table—and finally stacks the folded towel in a target location. The reliability and robustness of our algorithm enables for the first time a robot with general purpose manipulators to reliably and fully-autonomously fold previously unseen towels, demonstrating success on all 50 out of 50 single-towel trials as well as on a pile of 5 towels. I.

Abstract — We introduce a learning-based approach to manipulation in unstructured environments. This approach permits autonomous acquisition of manipulation expertise from interactions with the environment. The resulting expertise enables a robot to perform effective manipulation based on partial state information. The manipulation expertise is represented in a relational state representation and learned using relational reinforcement learning. The relational representation renders learning tractable by collapsing a large number of states onto a single, relational state. The relational state representation is carefully grounded in the perceptual and interaction skills of the robot. This ensures that symbolically learned knowledge remains meaningful in the physical world. We experimentally validate the proposed learning approach on the task of manipulating an articulated object to obtain a model of its kinematic structure. Our experiments demonstrate that the manipulation expertise acquired by the robot leads to substantial performance improvements. These improvements are maintained when experience is applied to previously unseen objects. I.

Abstract While complex hands seem to offer generality, simple hands are more practical for most robotic and telerobotic manipulation tasks, and will remain so for the foreseeable future. This raises the question: how do generality and simplicity trade off in the design of robot hands? This paper explores the tension between simplicity in hand design and generality in hand function. It raises arguments both for and against simple hands; it considers several familiar examples; and it proposes a concept for a simple hand design with associated strategies for grasping and object localization. The central idea is to use knowledge of stable grasp poses as a cue for object localization. This leads to some novel design criteria, such as a desire to have only a few stable grasp poses. We explore some of the design implications for a binpicking task, and then examine some experimental results to see how this approach might be applied in an assistive object retrieval task. 1

We present a generic framework that combines Dynamical Systems movement control with Programming by Demonstration (PbD) to teach a robot bimanual coordination task. The model consists of two systems: a learning system that processes data collected during the demonstration of the task to extract coordination constraints and a motor system that reproduces the movements dynamically, while satisfying the coordination constraints learned by the first system. We validate the model through a series of experiments in which a robot is taught bimanual manipulatory tasks with the help of a human. Categories and Subject Descriptors

Summary. We present our vision-based system for grasping novel objects in cluttered environments. Our system can be divided into four components: 1) decide where to grasp an object, 2) perceive obstacles, 3) plan an obstacle-free path, and 4) follow the path to grasp the object. While most prior work assumes availability of a detailed 3-d model of the environment, our system focuses on developing algorithms that are robust to uncertainty and missing data, which is the case in real-world experiments. In this paper, we test our robotic grasping system using our STAIR (STanford AI Robots) platforms on two experiments: grasping novel objects and unloading items from a dishwasher. We also illustrate these ideas in the context of having a robot fetch an object from another room in response to a verbal request. 1

Abstract — We consider the problem of grasping novel objects and its application to cleaning a desk. A recent successful approach applies machine learning to learn one grasp point in an image and a point cloud. Although those methods are able to generalize to novel objects, they yield suboptimal results because they rely on motion planner for finger placements. In this paper, we extend their method to accommodate grasps with multiple contacts. This approach works well for many humanmade objects because it models the way we grasp objects. To further improve the grasping, we also use a method that learns the ranking between candidates. The experiments show that our method is highly effective compared to a state-of-the-art competitor. I.

Abstract A simple hand is a robotic gripper that trades off generality in function for practicality in design and control. The long-term goal of our work is to explore that tradeoff and demonstrate broad manipulation capabilities with simple hands. This paper describes two prototype simple hands. Both hands have thin cylindrical fingers arranged symmetrically around a low friction circular palm. The fingers are compliantly coupled to a single actuator. Our experiments with both hands in a binpicking scenario demonstrate that we can achieve robust grasp classification and in-hand localization using simple statistical techniques. We further show how the classification accuracy increases as the grasp proceeds by exploiting information obtained online. We finally evaluate the relative importance of observing the full state of the hand rather than just observing the state of the actuators. 1

We believe that one of the major impediments to involvement in the field of robotics and AI is the difficulty end-users face in creating viable robot control policies. We seek to address this difficulty with lifelong robot learning and have developed an intuitive robot control interface using the Nintendo Wii remote to aid in this task. From three large public demos and several smaller ones, we have gotten a multitude of positive responses on the interface. We also believe that others can find similar successes in the field of HRI using undergraduate researchers.

Abstract — Handheld manipulable objects can often be found on flat surfaces within human environments. Researchers have previously demonstrated that perceptually segmenting a flat surface from the objects resting on it can enable robots to pick and place objects. However, methods for performing this segmentation can fail when applied to scenes with natural clutter. For example, low-profile objects and dense clutter that obscures the underlying surface can complicate the interpretation of the scene. As a first step towards characterizing the statistics of real-world clutter in human environments, we have collected and hand labeled 104 scans of cluttered tables using a tilting laser range finder (LIDAR) and a camera. Within this paper, we describe our method of data collection, present notable statistics from the dataset, and introduce a perceptual algorithm that uses machine learning to discriminate surface from clutter. We also present a method that enables a humanoid robot to place objects on uncluttered parts of flat surfaces using this perceptual algorithm. In cross-validation tests, the perceptual algorithm achieved a correct classification rate of 78.70 % for surface and 90.66 % for clutter, and outperformed our previously published algorithm. Our humanoid robot succeeded in 16 out of 20 object placing trials on 9 different unaltered tables, and performed successfully in several high-clutter situations. 3 out of 4 failures resulted from placing objects too close to the edge of the table. I.