. Visual servoing, using image-based control or positionbased control, generally gives satisfactory results. However, in some cases, convergence and stability problems may occur. The aim of this paper is to emphasize these problems by considering an eye-in-hand system and a positioning task with respect to a static target which constrains the six camera degrees of freedom. To appear in: The Confluence of Vision and Control, Lecture Notes in Control and Informations Systems, Springer-Verlag, 1998. 1 Introduction The two classical approaches of visual servoing (that is image-based control and position-based control) are different in the nature of the inputs used in their respective control schemes [28,10,14]. Even if the resulting robot behaviors thus also differ, both approaches generally give satisfactory results: the convergence to the desired position is reached, and, thanks to the closed-loop used in the control scheme, the system is stable, and robust with respect to camera calib...

SSD-based feature trackers have enjoyed growing popularity in recent years, particularly in the eld of visual servo control of robotic manipulators. These trackers use a sum-of-squared-dierences correlation measure to locate target features in a sequence of images. The results can then be used to estimate the motion of objects in the scene, to infer the 3D structure of the scene, or to control robot motions. The reliability of the information provided by these trackers can be degraded by a variety of factors, including changes in illumination, poor image contrast, occlusion of features, or unmodeled changes in objects. This has led other researchers to develop condence measures that are used to either accept or reject individual features that are located by the tracker. In this paper, we derive quantitative measures for the spatial uncertainty of the results provided by SSD-based feature trackers. Unlike previous condence measures which have been used only to accept or reject hypo...

We describe in this paper potential problems that may appear in image-based visual servoing when the initial camera position is far away from its desired position. We show by concrete examples that local minima or a singularity of the image Jacobian can be reached during the servoing. We then recall recent results obtained to avoid these drawbacks. It consists in combining visual features obtained directly from the image, and position-based features. This approach, called 2 1/2 D visual servoing, also provides supplementary advantages in function of the mean the features are combined with.

Abstract—In this paper, we present methods for tracking complex, articulated objects. We assume that an appearance model and the kinematic structure of the object to be tracked are given, leading to what is termed a model-based object tracker. At each time step, this tracker observes a new monocular grayscale image of the scene and combines information gathered from this image with knowledge of the previous configuration of the object to estimate the configuration of the object at the time the image was acquired. Each degree of freedom in the model has an uncertainty associated with it, indicating the confidence in the current estimate for that degree of freedom. These uncertainty estimates are updated after each observation. An extended Kalman filter with appropriate observation and system models is used to implement this updating process. The methods that we describe are potentially beneficial to areas such as automated visual tracking in general, visual servo control, and human computer interaction. Index Terms—Kalman filtering, object tracking. I.

We describe a model-based object tracking system that updates the configuration parameters of an object model based upon information gathered from a sequence of monocular images. Realistic object and imaging models are used to determine the expected visibility of object features, and to determine the expected appearance of all visible features. We formulate the tracking problem as one of parameter estimation from partially observed data, and apply the Extended Kalman Filtering (EKF) algorithm. The models are also used to determine what point feature movement reveals about the configuration parameters of the object. This information is used by the EKF to update estimates for parameters, and for the uncertainty in the current estimates, based on observations of point features in monocular images. 1

Performance estimation for feature tracking is a critical issue, if feature tracking results are to be used intelligently. In this paper, we derive quantitative measures for the spatial accuracy of a particular feature tracker. This method uses the results from the sum-of-squared-differences correlation measure commonly used for feature tracking to estimate the accuracy (in the image plane) of the feature tracking result. In this way, feature tracking results can be analyzed and exploited to a greater extent without placing undue confidence in inaccurate results or throwing out accurate results. We argue that this interpretation of results is more flexible and useful than simply using a confidence measure on tracking results to accept or reject features. For example, an extended Kalman filtering framework can assimilate these tracking results directly to monitor the uncertainty in the estimation process for the state of an articulated object. 1 Introduction Estimating the effectivene...

Abstract: The ability to analyze individual cells rather than averaged properties over a population is a major step towards understanding the fundamental elements of biological systems. Recent advances in microbiology such as cloning demonstrate that increasingly complex micromanipulation strategies for manipulating individual biological cells are required. In this paper, a microrobotic system capable of conducting automatic embryo pronuclei DNA injection is presented. Both embryo pronuclei DNA injection and intracytoplasmic injection (cell injection) are methods of introducing foreign genetic material into cells. Conventionally, cell injection has been conducted manually, however, long training, disappointingly low success rates from poor reproducibility in manual operations, and contamination all call for the elimination of direct human involvement. The system presented is capable of performing automatic embryo pronuclei DNA injection autonomously and semi-autonomously through a hybrid visual servoing control scheme. MEMS-based cell holders were designed and fabricated to aid in injection. Upon the completion of injection, the DNA injected embryos were transferred into a pseudopregnant foster female mouse to reproduce transgenic mice for cancer studies. Experiment result shows that the injection success rate is 100%. 1.

We propose a new near-real time technique for 3D face pose tracking from a monocular image sequence obtained from an uncalibrated camera. The basic idea behind our approach is that instead of treating 2D face detection and 3D face pose estimation separately, we perform simultaneous 2D face detection and 3D face pose tracking. Specifically, 3D face pose at a time instant is constrained by the face dynamics using Kalman Filtering and by the face appearance in the image. The use of Kalman Filtering limits possible 3D face poses to a small range while the best matching between the actual face image and the projected face image allows to pinpoint the exact 3D face pose. Face matching is formulated as an optimization problem so that the exact face location and 3D face pose can be estimated e#ciently. Another major feature of our approach lies in the use of active IR illumination, which allows to robustly detect eyes. The detected eyes can in turn constrain the face in the image and regularize the 3D face pose, therefore the tracking drift issue can be avoided and the processing can speedup. Finally, the face model is dynamically updated to account for variations in face appearances caused by face pose, face expression, illumination and the combination of them. Compared with

The robustness of visual tracking, or following the movement of objects in images, can be improved with an explicit model for the objects being tracked. In this paper, we investigate the use of an object model in this way. An object geometric model can tell us what feature movements to expect, and what those movements reveal about object motion. We characterize the tracking problem as one of parameter estimation from incomplete feature tracking data, and apply the Extended Kalman Filtering algorithm to the situation. Having an object model integrated into the tracking system overconstrains feature trackers, so that erroneous tracking results are selectively ignored and feasible tracking results are used to optimally update the object configuration estimate. 1 Introduction Visual object tracking is the following of the movements of objects in a scene by analysis of computer images of that scene. The tracking of features on an object is a necessary precursor to many tasks in vision-bas...

We describe in this paper potential problems that may appear in image-based visual servoing when the initial camera position is far away from its desired position. We show by concrete examples that local minima or a singularity of the image Jacobian can be reached during the servoing. We then recall recent results obtained to avoid these drawbacks. It consists in combining visual features obtained directly from the image, and position-based features. This approach, called 2 1/2 D visual servoing, also provides supplementary advantages in function of the mean the features are combined with.