While different optical flow techniques continue to appear, there has been a lack of quantitative evaluation of existing methods. For a common set of real and synthetic image sequences, we report the results of a number of regularly cited optical flow techniques, including instances of differential, matching, energy-based and phase-based methods. Our comparisons are primarily empirical, and concentrate on the accuracy, reliability and density of the velocity measurements; they show that performance can differ significantly among the techniques we implemented.

This paper presents a new approach to the estimation of two-dimensional motion vector fields from time-varying images. The approach is stochastic, both in its formulation and in the solution method. The formulation involves the specification of a deterministic structural model, along with stochastic observation and motion field models. Two motion models are proposed: a globally smooth model based on vector Markov random fields and a piecewise smooth model derived from coupled vector-binary Markov random fields. Two estimation criteria are studied. In the Maximum A Posteriori Probability (MAP) estimation the a posteriori probability of motion given data is maximized, while in the Minimum Expected Cost (MEC) estimation the expectation of a certain cost function is minimized. The MAP estimation is performed via simulated annealing, while the MEC algorithm performs iteration-wise averaging. Both algorithms generate sample fields by means of stochastic relaxation implemented via the Gibbs s...

Most robotic grasping tasks assume a stationary or fixed object. In this paper, we explore the requirements for tracking and grasping a moving object. The focus of our work is to achieve a high level of interaction between a real-time vision system capable of tracking moving objects in 3-D and a robot arm equipped with a dexterous hand that can be used pick up a moving object. We are interested in exploring the interplay of hand-eye coordination for dynamic grasping tasks such as grasping of parts on a moving conveyor system, assembly of articulated parts or for grasping from a mobile robotic system. Coordination between an organism's sensing modalities and motor control system is a hallmark of intelligent behavior, and we are pursuing the goal of building an integrated sensing and actuation system that can operate in dynamic as opposed to static environments. The system we have built addresses three distinct problems in robotic hand-eye coordination for grasping moving objects: fast computation of 3-d motion parameters from vision, predictive control of moving robotic arm to track a moving oblect, and grasp planning. The system is able to operate at approximately human arm movement rates, and we present experimenatl result in which a moving model train is tracked, stably grasped, and picked up by the system. The algorithms we have developed that relate sensing to actuation are quite general and applicable to a variety of complex robotic tasks that require visual feedback for arm and hand control.

In numerous domains it is useful to represent a single example by the set of the local features or parts that comprise it. However, this representation poses a challenge to many conventional machine learning techniques, since sets may vary in cardinality and elements lack a meaningful ordering. Kernel methods can learn complex functions, but a kernel over unordered set inputs must somehow solve for correspondences—generally a computationally expensive task that becomes impractical for large set sizes. We present a new fast kernel function called the pyramid match that measures partial match similarity in time linear in the number of features. The pyramid match maps unordered feature sets to multi-resolution histograms and computes a weighted histogram intersection in order to find implicit correspondences based on the finest resolution histogram cell where a matched pair first appears. We show the pyramid match yields a Mercer kernel, and we prove bounds on its error relative to the optimal partial matching cost. We demonstrate our algorithm on both classification and regression tasks, including object recognition, 3-D human pose inference, and time of publication estimation for documents, and we show that the proposed method is accurate and significantly more efficient than current approaches.

An eye-in-hand system visually tracking objects can fail when the manipulator encounters a kinematic singularity or a joint limit. A solution to this problem is presented in which objects are visually tracked while the manipulator simultaneously avoids kinematic singularities and manipulator joint limits by moving in directions along which the tracking task space is unconstrained or redundant. A manipulability measure is introduced into the visual tracking objective function, providing an elegant and robust technique for deriving a control law that visually tracks objects while accounting for the configuration of the manipulator. Two different tracking strategies, one using a standard visual tracking strategy and the other using the newly proposed strategy, are experimentally compared on an actual hand/eye system. The experimental results demonstrate the effectiveness of the new method by showing that the tracking region of a manipulator tracking objects with planar motion can be great...

We present a working implementation of a dynamics based architecture for visual sensing. This architecture provides field rate estimates of the positions and velocities of two independent falling balls in the face of repeated visual occlusions and departures from field of view. The practical success of this system can be attributed to the feedback interconnection between two strongly nonlinear dynamical systems: a novel "triangulating" state estimator; and an image plane window controller. We detail the architecture of this active sensor, provide data documenting its performance, and offer an initial analysis of its soundness in the form of a convergence proof for (a simpler version of) the estimator and a boundedness proof for (a somewhat idealized version of) the manager. 1 Introduction We have built a three degree of freedom robot that bats two balls into simultaneous stable periodic vertical trajectories that commonly persist for the better part of an hour [20]. The juggling algor...

This paper describes a new real-time tracking algorithm in conjunction with a predictive filter to allow real-time visual servoing of a robotic arm that is following a moving object. The system consists of two calibrated (but unregistered) cameras that provide images to a real-time, pipelined-parallel optic-flow algorithm that can robustly compute optic-flow and calculate the 3-D position of a moving object at approximately 5 Hz rates. These 3-D positions of the moving object serve as input to a predictive kinematic control algorithm that uses an a - b - g filter to update the position of a robotic arm tracking the moving object. Experimental results are presented for the tracking of a moving model train in a variety of different trajectories. 1. INTRODUCTION Tracking the three-dimensional movement of objects by a vision system in real-time is an important problem. It has been addressed by researchers in a number of different fields including target tracking, surveillance, automated ...

Traditional optical flow algorithms assume local image translational motion and apply simple image filtering. Recent studies have taken two separate approaches toward improving the accuracy of computed flow: the application of spatio-temporal filtering schemes and the use of generalized motion models such as the affine model. Each has achieved some improvement over traditional algorithms in specialized situations. In this paper, we analyze the interdependency between them and propose a unified approach. The general motion model we adopt characterizes arbitrary 3-D steady motion. Under perspective projection, we derive an image motion equation that describes the spatio-temporal relation of gray-scale intensity in an image sequence, thus making the utilization of 3-D filtering possible. However, to accommodate this complex motion, we need to extend the filter design to derive additional motion constraint equations. Using Hermite polynomials, we design differentiation filters, whose orthogonality and Gaussian derivative properties insure numerical stability. The resulting algorithm produces accurate optical flow and other useful motion parameters. It is evaluated quantitatively using the scheme established by Barron, et al.[4] and qualitatively with real images. 1.

Recovering structure from motion even using information from multiple image frames is difficult, in part because motion error can introduce large, correlated errors in the structure estimate. A method is proposed for recursively recovering structure from motion that can deal with this problem. Encouraging results on real images and synthetic data are presented. 1 Introduction Two frame structure from motion is known to be inaccurate. The reasons for this are familiar: for two frames, each structure value is determined by a single measurement which is unreliable for distant points, and which is strongly affected by small errors in the motion. The natural remedy to this problem is to refine the structure estimation by combining measurements from many frames [2] [3] [8] [11] [12] [13] [16]. In earlier work [14] [18], we have presented an implementation of this procedure which differs from previous approaches in that the motion error is explicitly taken into account. In this paper, a det...

We argue that intelligence is necessary in robots used for rehabilitation in order to reduce the amount of mental activity needed by the user of these robots. With this in mind, the areas of research relevant to imparting robotic systems with the capability of assuming a more intelligent role are identified. We describe our implementation of functionalities such as fuzzy command interpretation, object recognition, face tracking, and task planning and learning, which are part of the ISAC, an intelligent system designed to feed individuals with physical disabilities. I. Introduction Intelligence in a rehabilitation robotic system is not usually stressed because of the presence of an intelligent person (the user) who is expected to make all the decisions requiring intelligence. This expectation has resulted in the development of many rehabilitation robots that provide an interface for the user to issue robot motion commands by means of teleoperation [1]. Although teleoperation may reliev...