We propose two methods for visual control of a robotic system which do not require the formulation of an explicit calibration between image space and the world coordinate system. Calibration is known to be a difficult and error prone process. By extracting control information directly from the image, we free our techniques from the errors normally associated with a fixed calibration. The two algorithms we propose both utilize feedback from a simple geometric effect, rotational invariance, to control the positioning servo loop. We attach a camera system to a robot such that the camera system and the robot's gripper rotate simultaneously. We also constrain the camera to lie in a position where it can observe the gripper's rotational axis. As the camera system rotates about the gripper's rotational axis, the circular path traced out by a point-like feature projects to an elliptical path in image space. We gather the projected feature points over part of a rotation (=2 radians) ...

Most robotic hands are either sensorless or lack the ability to accurately and robustly report position and force information relating to contact. This paper describes a robotic hand system that uses a limited set of native joint position and force sensing along with custom designed tactile sensors and real-time vision modules to accurately compute finger contacts and applied forces for grasping tasks. Three experiments are described: integration of real-time visual trackers in conjunction with internal strain gauge sensing to correctly localize and compute finger forces, determination of contact points on the inner and outer links of a finger through tactile sensing and visual sensing, and determination of vertical displacementby tactile sensing for a grasping task.

Most robotic hands are either sensorless or lack the ability to report accurate position and force information relating to contact. This paper describes a robotic hand system that uses a limited set of native joint position and force sensing along with custom designed tactile sensors and real-time vision modules to accurately compute finger contacts and applied forces for grasping tasks. A number of experiments are described that show how these sensors can be combined to increase the capabilities of a robotic hand for grasping and manipulation tasks. 1

This paper discusses the problem of visual control of grasping. We have implemented an object tracking system that can be used to provide visual feedback for locating the positions of fingers and objects, as well as the relative relationships between them. This visual analysis can be used to control open loop grasping systems where finger contact, object movement, and task completion need to be monitored and controlled. 1 Introduction Hand-eye coordination is a demanding robotic task. While such coordination is common in higher level animals, it is still an elusive goal for multi-sensor robots. A key component of this is to fuse both vision and touch information in real-time. Such fusion will allow on-line grasp planning and manipulation. This paper addresses the vision aspect of this sensor fusion problem, specifically integrating visual feedback for control of grasping and manipulation. We motivate the application of vision to grasping with the following example taken from manufact...

To perform dextrous manipulation efficiently, it is necessary to coordinate the interactions of many component processes. This paper investigates one approach to coordination: discrete-event systems. The applicability of discrete-event systems to the modeling of dextrous manipulation tasks is studied. Discrete-event control theory offers formal methods for determining whether a coordinator of the components can be generated. A representative dextrous manipulation task, the planar Grasp-Lift-Replace task of Howe and Cutkosky, is presented as a discrete-event process. The task is extended to include two-fingered exploratory procedures. The effectiveness of the discrete-event system approach is illustrated through simulations of several test cases. 1 Introduction In many robotics problems, robot hands must perform delicate and precise operations that include grasping and smoothly lifting an object. Robotics research has addressed many components of this dextrous manipulation problem. The...

We describ e a system that integrates real-time computer vision with a sensorless gripper to provide closed loop feedback control for grasping and manipulation tasks. Many hand-eye coordination skills can be thought of as sensory-control loops, where specialized reasoning has been embodied as a feedback or control path in the loop's construction. Our framework captures the essence of these hand-eye coordination skills in simple visual control primitives, which are a key component of the software integration. The primitives use a simple visual tracking and correspondence scheme to provide real-time feedback control in the presence of imprecise camera calibrations. Experimental results are shown for the positioning task of locating, picking up, and inserting a bolt into a nut under visual control. Results are alsopresented for the visual control of abolt tightening task. 1

Sensors play an important role in providing feedback to robotic control systems. Vision can be an effective sensing modality due to its speed, low cost, and flexibility. It can also serve as an external sensor that can provide control information for devices that lack internal sensing. Many robot grippers do not have internal sensors. These grippers can be prone to error as they operate under open loop control and usually require a precise model of the environment to be effective. This paper describes 2 visual control primitives that can be used to provide position control for a sensorless robot system. The primitives use a simple visual tracking and correspondence scheme to provide real-time feedback control in the presence of imprecise camera calibrations. Experimental results are shown for the positioning task of locating, picking up, and inserting a bolt into a nut under visual control. Results are also presented for the visual control of a bolt tightening task. 1 Introduction In ...