To describe phenomena that occur at different time scales, computational models of the brain must necessarily incorporate different levels of abstraction. We argue that at time scales of approximately one-third of a second, orienting movements of the body play a crucial role in cognition and form a useful computational level, termed the embodiment level . At this level, the constraints of the body determine the nature of cognitive operations, since the natural sequentiality of body movements can be matched to the natural computational economies of sequential decision systems. The way this is done is through a system of implicit reference termed deictic, whereby pointing movements are used to bind objects in the world to cognitive programs. We show how deictic bindings enable the solution of natural tasks and argue that one of the central features of cognition, working memory, can be related to moment-by-moment dispositions of body features such as eye movements and hand movements. Keyw...

Most robotic grasping tasks assume a stationary or fixed object. In this paper, we explore the requirements for tracking and grasping a moving object. The focus of our work is to achieve a high level of interaction between a real-time vision system capable of tracking moving objects in 3-D and a robot arm equipped with a dexterous hand that can be used pick up a moving object. We are interested in exploring the interplay of hand-eye coordination for dynamic grasping tasks such as grasping of parts on a moving conveyor system, assembly of articulated parts or for grasping from a mobile robotic system. Coordination between an organism's sensing modalities and motor control system is a hallmark of intelligent behavior, and we are pursuing the goal of building an integrated sensing and actuation system that can operate in dynamic as opposed to static environments. The system we have built addresses three distinct problems in robotic hand-eye coordination for grasping moving objects: fast computation of 3-d motion parameters from vision, predictive control of moving robotic arm to track a moving oblect, and grasp planning. The system is able to operate at approximately human arm movement rates, and we present experimenatl result in which a moving model train is tracked, stably grasped, and picked up by the system. The algorithms we have developed that relate sensing to actuation are quite general and applicable to a variety of complex robotic tasks that require visual feedback for arm and hand control.

Conventional methods for programming a robot either are inflexible or demand significant expertise. While the notion of automatic programming by high-level goal specification addresses these issues, the overwhelming complexity of planning manipulator grasps and paths remains a formidable obstacle to practical implementation. Our approach of programming a robot is by direct human demonstration. Our system observes a human performing the task, recognizes the human grasp, and maps it onto the manipulator. Using human actions to guide robot execution greatly reduces the planning complexity. Subsequent to recording the human task execution, temporal task segmentation is carried out to identify task breakpoints. This step facilitates human grasp recognition and object motion extraction for robot execution of the task. This paper describes how an observed human grasp can be mapped to that of a given general-purpose manipulator for task replication. Planning the manipulator grasp based upon the observed human grasp is done at two levels: the functional and physical levels. Initially, at the functional level, grasp mapping is achieved at the virtual finger level; the virtual finger is a group of fingers acting against an object surface in a similar manner. Subsequently, at the physical level, the geometric properties of the object and manipulator are considered in finetuning the manipulator grasp. Our work concentrates on power or enveloping grasps and the fingertip precision grasps. We conclude by showing an example of an entire programming cycle from human demonstration to robot execution.

This article describes a theory of the computations underlying the selection of coordinated motion patterns, especially in reaching tasks. The central idea is that when a spatial target is selected as an object to be reached, stored postures are evaluated for the contributions they can make to the task. Weights are assigned to the stored postures, and a single target posture is found by taking a weighted sum of the stored postures. Movement is achieved by reducing the distance between the starting angle and target angle of each joint. The model explains compensation for reduced joint mobility, tool use, practice effects, performance errors, and aspects of movement kinematics. Extensions of the model can account for anticipation and coarticulation effects, movement through via points, and hierarchical control of series of movements. The goal of this research is a unified theory of the planning and control of physical action. Such a theory, as several authors have noted (Jeannerod, in press; Rosenbaum, 1991; Wing, 1993), has been lacking. Instead, specialized models have been designed to account for data from different tasks. The sentiment

Conventional methods for programming a robot either are inflexible or demand significant expertise. While the notion of automatic programming by high-level goal specification addresses these issues, the overwhelming complexity of planning manipulator grasps and paths remains a formidable obstacle to practical implementation. This thesis describes the approach of programming a robot by human demonstration. Our system observes a human performing the task, recognizes the human grasp, and maps it onto the manipulator. Using human actions to guide robot execution greatly reduces the planning complexity. In analyzing the task sequence, the system first divides the observed sensory data into meaningful temporal segments, namely the pregrasp, grasp, and manipulation phases. This is achieved by analyzing the human hand motion profiles. The features used are the fingertip polygon area (the fingertip polygon being the polygon whose vertices are the fingertips), hand speed, and the volume sweep r...

In the computer animation field, the interest for grasping has appeared with the development of synthetic actors. Based on a grasp taxonomy, we propose a completely automatic grasping system for synthetic actors. In particular, the system can decide to use a pinch when the object is too small to be grasped by more than two fingers or to use a two-handed grasp when the object is too large. The system also offers both direct and inverse kinematics to control the articulations. In order to ensure realistic looking closing of the hand, several of the joints are constrained. A brief description of the system and results are also presented.

Simulating a human figure performing a task requires that the agent interact with objects in the environment in a realistic manner. In this paper we describe a system which directs task-level, general-purpose, object grasping for a simulated human agent. The Object Specific Reasoner (OSR) generates parameters for underspecified task-level instructions such as (pickup jack hammer). The Grasp behavior manages simultaneous motions of the joints in the hand, wrist and arm. When composed hierarchically, the OSR and the Grasp behavior interpret task-level commands to the animation system. These modules are implemented as part of the Jack project at the University of Pennsylvania.

Abstract not found

We present a simple sign language for teleassistance inspired by the work of the physiologist Nicolai Bernstein and by recent psychophysical evidence in hand-eye coordination. In our approach, a teleoperator uses hand signs to guide an otherwise autonomous robot manipulator through a given task. Each sign signals a context switch and provides a hand-centered reference frame for the robot's servomotor routines. The signs are natural, such as pointing to an object to indicate the desire to reach toward it as well as the axis along which to reach. Following the lead of [Agre & Chapman 1987] we term these signs deictic from the Greek word for pointing to stress their indicative and relative nature. The task example is opening a door using a Utah/MIT hand mounted on a Puma 760 arm. The teleoperator wears an EXOS hand master and polhemus sensor. Three variations of nearest neighbor pattern classification are tested for online recognition of the sign language. The simplest, in which the oper...

This paper describes an implementation of a behaviorbased mobile manipulator capable of autonomously transferring a sample from one drum to a second in unstructured environments. A major contribution of the project was the coherent integration of the arm and base as a cohesive unit, and not just a mobile base with an arm attached. The support for smooth simultaneous operation of all joints on the vehicle facilitated biologically plausible motions, such as arm preshaping. The behavior-based controller used a pseudo-force model, where behaviors add forces and torques to joints and limbs resulting in coordinated motion. The vehicle Jacobian is used to convert the pseudo-forces into joint torques and a pseudo-damping model converts the joint torques into joint velocities. This process allows rapid control of the manipulator without the use of inverse kinematics. A drum sampling task is presented where the vehicle demonstrates how a sample of material could be moved from one drum to another...