. Visual servoing, using image-based control or positionbased control, generally gives satisfactory results. However, in some cases, convergence and stability problems may occur. The aim of this paper is to emphasize these problems by considering an eye-in-hand system and a positioning task with respect to a static target which constrains the six camera degrees of freedom. To appear in: The Confluence of Vision and Control, Lecture Notes in Control and Informations Systems, Springer-Verlag, 1998. 1 Introduction The two classical approaches of visual servoing (that is image-based control and position-based control) are different in the nature of the inputs used in their respective control schemes [28,10,14]. Even if the resulting robot behaviors thus also differ, both approaches generally give satisfactory results: the convergence to the desired position is reached, and, thanks to the closed-loop used in the control scheme, the system is stable, and robust with respect to camera calib...

In this paper, we propose a new approach to vision-based robot control, called 2 1/2 D visual servoing, which avoids the respective drawbacks of classical positionbased and image-based visual servoing. Contrary to the position-based visual servoing, our scheme does not need any geometric 3D model of the object. Furthermore and contrary to image-based visual servoing, our approach ensures the convergence of the control law in the whole task space. 2 1/2 D visual servoing is based on the estimation of the partial camera displacement from the current to the desired camera poses at each iteration of the control law. Visual features and data extracted from the partial displacement allow us to design a decoupled control law controlling the six camera d.o.f. The robustness of our visual servoing scheme with respect to camera calibration errors is also analyzed: the necessary and sufficient conditions for local asymptotic stability are easily obtained. Then, due to the simple structure of the ...

We introduce the problem of computing robot motion strategies that maintain visibility of a moving target in a cluttered workspace. Both motion constraints (as considered in standard motion planning) and visibility constraints (as considered in visual tracking) must be satisfied. Additional criteria, such as the total distance traveled, can be optimized. The general problem is divided into two categories, on the basis of whether the target is predictable. For the predictable case, an algorithm that computes optimal, numerical solutions is presented. For the more challenging case of a partially-predictable target, two on-line algorithms are presented that each attempt to maintain future visibility with limited prediction. One strategy maximizes the probability that the target will remain in view in a subsequent time step, and the other maximizes the minimum time in which the target could escape the visibility region. We additionally discuss issues resulting from our implementation and e...

This article is the first of a two-part series on the topic of visual servo control—using computer vision data in the servo loop to control the motion of a robot. In the present article, we describe the basic techniques that are by now well established in the field. We first give a general overview of the formulation of the visual servo control problem. We then describe the two archetypal visual servo control schemes: image-based and position-based visual servo control. Finally, we discuss performance and stability issues that pertain to these two schemes, motivating the second article in the series, in which we consider advanced techniques.

Active cameras provide a navigating vehicle with the ability to fixate and track features over extended periods of time, and wide fields of view. While it is relatively straightforward to apply fixating vision to tactical, short-term navigation tasks, using serial fixation on a succession of features to provide global information for strategic navigation is more involved. However, active vision is seemingly well-suited to this task: the ability to measure features over such a wide range means that the same ones can be used as a robot makes a wide range of movements. This has advantages for map-building and localisation. The core work of this thesis concerns simultaneous localisation and map-building for a robot with a stereo active head, operating in an unknown environment and using point features in the world as visual landmarks. Importance has been attached to producing maps which are useful for extended periods of navigation. Many map-building methods fail on extended runs because ...

We present an original method for tracking, in an image sequence, complex objects which can be approximately modeled by a polyhedral shape. The approach relies on the estimation of the 2D object image motion along with the computation of the 3D object pose. The proposed method fulfills real-time constraints along with reliability and robustness requirements. Real tracking experiments and results concerning a visual servoing positioning task are presented.

Tracking is a very important research subject in a real-time augmented reality context. The main requirements for trackers are high accuracy and little latency at a reasonable cost. In order to address these issues, a real-time, robust, and efficient 3D modelbased tracking algorithm is proposed for a “video see through ” monocular vision system. The tracking of objects in the scene amounts to calculating the pose between the camera and the objects. Virtual objects can then be projected into the scene using the pose. Here, nonlinear pose estimation is formulated by means of a virtual visual servoing approach. In this context, the derivation of point-to-curves interaction matrices are given for different 3D geometrical primitives including straight lines, circles, cylinders, and spheres. A local moving edges tracker is used in order to provide real-time tracking of points normal to the object contours. Robustness is obtained by integrating an M-estimator into the visual control law via an iteratively reweighted least squares implementation. This approach is then extended to address the 3D model-free augmented reality problem. The method presented in this paper has been validated on several complex image sequences including outdoor environments. Results show the method to be robust to occlusion, changes in illumination, and mistracking.

Image-based control of free-flying mechanical systems is addressed, with application to an indoor, vision-guided blimp. A methodology is developed for performing visual servoing by incorporating the physical parameters of a mechanical system into the image plane dynamics. It is noted that under suitable conditions, namely the existence of a diffeomorphism between image features and the robot pose, many of the tools from mechanical systems control theory can be used directly when applied in the image plane. It is also shown that when a suitable Jacobian map exists between the pose and image feature spaces, dynamics for vision-based control of aerial and underwater vehicles can easily be formulated. Experimental results demonstrate initial success at spatial tracking of simple objects. 1 Introduction Visual servo control has rapidly emerged as an exciting new field of research. Since its debut in the 1980's, it has attracted increasing interest from both industry and academia. Early re...

In this paper, several vision-based robot control methods are classified following an analogy with well known minimization methods. Comparing the rate of convergence between minimization algorithms helps us to understand the difference of performance of the control schemes. In particular, it is shown that standard vision-based control methods have in general low rates of convergence. Thus, the performance of vision-based control could be improved using schemes which perform like the Newton minimization algorithm that has a high convergence rate. Unfortunately, the Newton minimization method needs the computation of second derivatives that can be ill-conditioned causing convergence problems. In order to solve these problems, this paper proposes two new control schemes based on efficient second-order minimization techniques.

In this paper, we determine the analytical form of the interaction matrix related to any moment that can be computed from segmented images. The derivation method we present is based on Green's theorem. We apply this general result to classical geometrical primitives. We then consider using moments in image-based visual servoing. For that, we select six combinations of moments to control the six degrees of freedom of the system. These features are particularly adequate, if we consider a planar object and the configurations such that the object and camera planes are parallel at the desired position. The experimental results we present show that a correct behavior of the system is obtained if we consider either a simple symmetrical object or a planar object with complex and unknown shape.