This paper provides a tutorial introduction to visual servo control of robotic manipulators. Since the topic spans many disciplines our goal is limited to providing a basic conceptual framework. We begin by reviewing the prerequisite topics from robotics and computer vision, including a brief review of coordinate transformations, velocity representation, and a description of the geometric aspects of the image formation process. We then present a taxonomy of visual servo control systems. The two major classes of systems, position-based and image-based systems, are then discussed. Since any visual servo system must be capable of tracking image features in a sequence of images, we include an overview of feature-based and correlation-based methods for tracking. We conclude the tutorial with a number of observations on the current directions of the research field of visual servo control. 1 Introduction Today there are over 800,000 robots in the world, mostly working in factory environment...

The possibility of calibrating a camera from image data alone, based on matched points identified in a series of images by a moving camera was suggested by Mayband and Faugeras. This result implies the possibility of Euclidean reconstruction from a series of images with a moving camera, or equivalently, Euclidean structure-frommotion from an uncalibrated camera. No tractable algorithm for implementing their methods for more than three images have been previously reported. This paper gives a practical algorithm for Euclidean reconstruction from several views with the same camera. The algorithm is demonstrated on synthetic and real data and is shown to behave very robustly in the presence of noise giving excellent calibration and reconstruction results. 1

Modelling th# push broom sensors commonly used in satellite imagery is quite di#cult and computationally intensive due to th# complicated motion ofth# orbiting satellite with respect to th# rotating earth# In addition, th# math#46 tical model is quite complex, involving orbital dynamics, andh#(0k is di#cult to analyze. Inth#A paper, a simplified model of apush broom sensor(th# linear push broom model) is introduced. Ith as th e advantage of computational simplicity wh#A9 atth# same time giving very accurate results compared with th# full orbitingpush broom model. Meth# ds are given for solving th# major standardph# togrammetric problems for th e linear push broom sensor. Simple non-iterative solutions are given for th# following problems : computation of th# model parameters from groundcontrol points; determination of relative model parameters from image correspondences between two images; scene reconstruction given image correspondences and ground-control points. In addition, th# linearpush broom model leads toth#0 retical insigh ts th# t will be approximately valid for th# full model as well.Th# epipolar geometry of linear push broom cameras in investigated and sh own to be totally di#erent from th at of a perspective camera. Neverth eless, a matrix analogous to th e essential matrix of perspective cameras issh own to exist for linear push broom sensors. Fromth#0 it is sh# wn th# t a scene is determined up to an a#ne transformation from two viewswith linearpush broom cameras. Keywords :push broom sensor, satellite image, essential matrixph# togrammetry, camera model The research describ ed in this paper hasb een supportedb y DARPA Contract #MDA97291 -C-0053 1 Real Push broom sensors are commonly used in satellite cameras, notably th# SPOT satellite forth# generatio...

The fundamental matrix is a basic tool in the analysis of scenes taken with two uncalibrated cameras, and the 8-point algoritm is a frequent#e cit#3 met#9 d for comput#10 t he fundament al ma t# ix from a set of 8 or more point mat ches. It hast he advant age of simplicit y of implement at ion. The prevailing view is, however,t#(9 it isext#3791( suscept#-43 t o noise and hence virtually useless for most purposes. This paper challengest#en view, by showing t#ng by precedingt he algorit hm wit h a very simple normalizat ion(t ranslat ion and scaling) oft he coordinat es oft he mat ched point#( result# are obt# ined comparable wit# t he best it## at ive algorit#209 This improved performance is just#690 byt#1082 and verified byext#259( e experiment s on real images.

This paper attempts to present a comprehensive summary of research results in the use of visual information to control robot manipulators and related mechanisms. An extensive bibliography is provided which also includes important papers from the elemental disciplines upon which visual servoing is based. The research results are discussed in terms of historical context, commonality of function, algorithmic approach and method of implementation. 1 Introduction This paper presents the history, and reviews current research into the use of visual information for the control of robot manipulators and mechanisms. Visual control of manipulators promises substantial advantages when working with targets whose position is unknown, or with manipulators which may be flexible or inaccurate. The reported use of visual information to guide robots, or more generally mechanisms, is quite extensive and encompasses manufacturing applications, teleoperation, missile tracking cameras, fruit picking as well...

This paper gives a new method for image rectification, the process of resampling pairs of stereo images taken from widely differing viewpoints in order to produce a pair of "matched epipolar projections". These are projections in which the epipolar lines run parallel with the x-axis and consequently, disparities between the images are in the x-direction only. The method is based on an examination of the essential matrix of Longuet-Higgins which describes the epipolar geometry of the image pair. The approach taken is consistent with that recently advocated strongly by Faugeras ([1]) of av oiding camera calibration. The paper uses methods of projective geometry to define a matrix called the "epipolar transformation matrix" used to determine a pair of 2D projective transforms to be applied to the two images in order to match the epipolar lines. The advantages include the simplicity of the 2D projective transformation which allows very fast resampling as well as subsequent simplification in the identification of matched points and scene reconstruction.

oints. This new al gorithm integrates bothl ines and points into onel inear framework. This contrasts with previousl y known al gorithms that have been specific to either point sets (for exampl# the 8-pointal#886P3S of [5, 6]) orl#85 sets ([3, 4]). Previousl# to deal with both points andl#360 at the same time, an iterativel east-squares approach has been necessary. The newal#8z8PN3 has not to date been tried on mixed sets of points andl#d3P5 However, the resul#= obtained in ([3, 4]) forl#5=fi and in [7] using points, both special cases of this al gorithm show quite good resul#5z Thanks are du# to Long Qu an and Andrew Zisserman for discu# sions before and du## ng the preparation of this note. 2Not at ion Al# vectors are written asl ower case bo l#l#5W58P3 such asu or #. Such a notation denotes a col umn vector. A row vector is written as a transposedcol umn vector such # .The inner product of two vectors is written as a # b, that is by viewing a vectors as a matrix with onl# one col

This paper describes a simple method for internal camera calibration for computer vision systems. It is intended for use with medium to wide angle camera lenses. With modification it can be used for longer focal lengths. This method is based on tracking image features through a sequence of images while the camera undergoes pure rotation. This method does not require a special calibration object. The location of the features relative to the camera or to each other need not be known. It is only required that the features can be located accurately in the image. This method can therefore be used both for laboratory calibration and for self calibration in autonomous robots working in unstructured environments. The method works when features can be located to single pixel accuracy, but subpixel accuracy should be used if available. In the

Existing linear solutions for the pose estimation (or exterior orientation) problem suffer from a lack of robustness and accuracy partially due to the fact that the majority of the methods utilize only one type of geometric entity and their frameworks do not allow simultaneous use of different types of features. Furthermore, the orthonormality constraints are weakly enforced or not enforced at all. We have developed a new analytic linear least-squares framework for determining pose from multiple types of geometric features. The technique utilizes correspondences between points, between lines, and between ellipse-circle pairs. The redundancy provided by different geometric features improves the robustness and accuracy of the least-squares solution. A novel way of approximately imposing orthonormality constraints on the sought rotation matrix within the linear framework is presented. Results from experimental evaluation of the new technique using both synthetic data and real images revea...

This paper desc#xw es an implementation of the Cubic Rational Polynomial Camera model developed as part of the FOCUS projec# . FOCUS ([1]) is on ongoing "shared vision" IR&D projec# jointly sponsored by Loc kheed Martin Missiles and Spac e (LMMSS/Sunnyvale) and General Elec# ric CR&D. A c# bic c amera has the advantage that allc ameras, suc h as projec tive, a#ne and the linear pushbroom, whic h map the image points as rational polynomial func tions (of degree no greater than 3) of thec oordinates of a world point,c## be treated as spec#wz c ases of the c ubic c amera. This paper demonstrates that thec ubic c amerac an very e#ec tively model even thosec ameras whic h express the image points as c#3 plic#, ed func tions of worldc# ordinates, suc h as radic#wzfi In partic# - lar, it is empiric##xA demonstrated that a SAR sensor is veryac#3 rately approximated by ac ubic c amera, but not by any linearc amera model. The paper also outlines an algorithm for estimating the parameters of thec ubic c amera, given a set of image to worldc orrespondenc es. The non-linear nature of thisc amerac an make parameter estimation a very unstable proc ess. The slightest noise in thec oe#c ients of the nonlinear termsc an lead to ac ompletely unrealistic model of thec amera. This paper disc usses some refinements suc h as avoiding degenerac ies, data normalization, and regularization whic h arenec#9 sary for ac#w#3 te estimation of the c# bic c amera parameters and minimization of noise in the c# e#c#[3 ts of the higher degree terms. 1 Introducti on A basic requ#- ement of the FOCUS ([1]) project is to be able to compu#/ camera models and do modelbu#II8&D u sing complexand general camera models. To thispu# pose, aCu#8# Rational Polynomial camera model has been developed in FOCUS to aid in...