Most robotic grasping tasks assume a stationary or fixed object. In this paper, we explore the requirements for tracking and grasping a moving object. The focus of our work is to achieve a high level of interaction between a real-time vision system capable of tracking moving objects in 3-D and a robot arm equipped with a dexterous hand that can be used pick up a moving object. We are interested in exploring the interplay of hand-eye coordination for dynamic grasping tasks such as grasping of parts on a moving conveyor system, assembly of articulated parts or for grasping from a mobile robotic system. Coordination between an organism's sensing modalities and motor control system is a hallmark of intelligent behavior, and we are pursuing the goal of building an integrated sensing and actuation system that can operate in dynamic as opposed to static environments. The system we have built addresses three distinct problems in robotic hand-eye coordination for grasping moving objects: fast computation of 3-d motion parameters from vision, predictive control of moving robotic arm to track a moving oblect, and grasp planning. The system is able to operate at approximately human arm movement rates, and we present experimenatl result in which a moving model train is tracked, stably grasped, and picked up by the system. The algorithms we have developed that relate sensing to actuation are quite general and applicable to a variety of complex robotic tasks that require visual feedback for arm and hand control.

The vast majority of work in machine vision emphasizes the representation of perceived objects and events: it is these internal representations that incorporate the `knowledge' in knowledge-based vision or form the `models' in model-based vision. In this paper, we discuss simple machine vision systems developed by artificial evolution rather than traditional engineering design techniques, and note that the task of identifying internal representations within such systems is made difficult by the lack of an operational definition of representation at the causal mechanistic level. Consequently, we

We present a model for recovering the direction of heading of an observer who is moving relative to a scene that may contain self-moving objects. The model builds upon an algorithm proposed by Rieger and Lawton (1985), which is based on earlier work by Longuet-Higgins and Prazdny (1981). The algorithm uses velocity differences computed in regions of high depth variation to estimate the location of the .focus o.f ezpansion, which indicates the observer's heading direction. We relate the behavior of the proposed model to psychophysical observations regarding the ability of human observers to judge their heading direction, and show how the model can cope with self- moving objects in the environment. We also discuss this model in the broader context of a navigational system that performs tasks requiring rapid sensing and response through the interaction of simple task-specific routines.

The aim of this study was to determine the role of head, eye and arm movements during the execution of a table tennis forehand stroke. Three-dimensional kinematic analysis of line-of-gaze, arm and ball was used to describe visual and motor behaviour. Skilled and less skilled participants returned the ball to cued right or left target areas under three levels of temporal constraint: pre-, early- and late-cue conditions. In the pre- and early-cue conditions, both high and low skill participants tracked the ball early in ¯ ight and kept gaze stable on a location in advance of the ball before ball ± bat contact. Skilled participants demonstrated an earlier onset of ball tracking and recorded higher performance accuracy than less skilled counterparts. The manipulation of cue condition showed the limits of adaptation to maintain accuracy on the target. Participants were able to accommodate the constraints imposed by the early-cue condition by using a shorter quiet eye duration, earlier quiet eye oþ set and reduced arm velocity at contact. In the late-cue condition, modi ® cations to gaze, head and arm movements were not suý cient to preserve accuracy. The ® ndings highlight the functional coupling between perception and action during time-constrained, goal-directed actions.

Organisms function in particular environments, the properties of which are reflected in the structure of their nervous systems. Therefore, sensory information may directly specify motor behaviors. We argue that such specification involves the coupling of sensory information into appropriately structured control systems that generate action. The nature of this coupling as well as the structure of the control systems reflect properties of the environment. This is most dramatically demonstrated when adaptive processes adjust the underlying control system in response to changes in the environment. Experimental and modelling work on posture in perturbed visual and haptic environments is reviewed to provide evidence for these arguments. Theoretical modelling and autonomous robotics work that goes beyond the posture example of perception-action coupling is briefly discussed, primarily in order to point out that the integration of multiple behavioral constraints is a non-trivial problem that h...

A theoretical model is proposed that attempts to take into account the contribution of the environmental surround to the perception of time to contact of an object approaching an observer or performer. The proposed model is based on the assumption of direct approach at constant velocity, as is the case for Lee's tau model (Lee, Young, Reddish, Lough & Clayton, 1983). However, our theoretical model incorporates optical information from both the approaching object (as in Lee's 'c model) and the surrounding environment (not included in Lee's q: model). This background optical information is contained in a new construct that we refer to as background tau ('Cbg). Both object tau ('obj) and 'Cbg are measured in terms of the rate of expansion (or reduction) of the areas within the appropriate closed contours on the retina. Relative tau ('OR) is defined in terms of'obj and 'Cbg and reflects the rate of expansion (or reduction) of the retinal image of the object relative to that of an adjacent, surrounding background patch. We argue that perceived time to contact of the object with the observer is dependent on 'R- Under conditions of a static environmental surround, our 'c R reduces to Lee's c. However, we argue that perception of the object is still made with respect to this surrounding static environment. Our model makes additional predictions with regard to perceived time to contact of the object with the observer when the environmental flow field moves either toward or away from the observer or performer. In this paper, we indicate the implications of the model for the production of an interceptive motor action such as catching or hitting a ball, and offer some suggestions on how the model can be experimentally tested. While the model can be considered a refinement of Lee's ...

... this paper, we discuss simple machine vision systems developed by artificial evolution rather than traditional engineering design techniques, and note that the task of identifying internal representations within such systems is made difficult by the lack of an operational definition of representation at the causal mechanistic level. Consequently,we question the nature and indeed the existence of representations posited to be used within natural vision systems (i.e., animals). We conclude that representations argued for on a priori grounds by external observers of a particular vision system may well be illusory, and are at best placeholders for yet-to-be-identified causal mechanistic interactions. That is, applying the knowledge-based vision approach in the understanding of evolved systems (machines or animals) maywell lead to theories and models which are internally consistent, computationally plausible, and entirely wrong.