This paper gives a new method for image rectification, the process of resampling pairs of stereo images taken from widely differing viewpoints in order to produce a pair of "matched epipolar projections". These are projections in which the epipolar lines run parallel with the x-axis and consequently, disparities between the images are in the x- direction only. The method is based on an examination of the fundamental matrix of Longuet-Higgins which describes the epipolar geometry of the image pair. The approach taken is consistent with that recently advocated by Faugeras ([1]) of avoiding camera calibration. The paper uses methods of projective geometry to determine a pair of 2D projective transformations to be applied to the two images in order to match the epipolar lines. The advantages include the simplicity of the 2D projective transformation which allows very fast resampling as well as subsequent simplification in the identification of matched points and scene reconstruction. 1 In...

We present an efficient and geometrically intuitive algorithm to reliably interpret the image velocities of moving objects in 3D. It is well known that under weak perspective the image motion of points on a plane can be characterised by an affine transformation. We show that the relative image motion of a nearby noncoplanar point and its projection on the plane is equivalent to motion parallax and because it is independent of viewer rotations it is a reliable geometric cue to 3D shape and viewer/object motion In particular we show how to interpret the motion parallax vector of non-coplanar points (and contours) and the curl, divergence and deformation components of the affine transformation (defined by the three points or a closed-contour of the plane) in order to recover the projection of the axis of rotation of a moving object; the change in relative position of the object; the rotation about the ray; the tilt of the surface and a one parameter family of solutions for the slant as a function of the magnitude of the rotation of the object. The latter is a manifestation of the bas–relief ambiguity. These measurements, although representing an incomplete solution to structure from motion, are the only subset of structure and motion parameters which can be reliably extracted from two views when perspective effects are small. We present a real-time example in which the 3D visual interpretation of hand gestures or a hand-held object is used as part of a man-machine interface. This is an alternative to the Polhemus coil instrumented Dataglove commonly used in sensing manual gestures. 1

In this article we provide a conceptual framework in which to think of the relationships between the three-dimensional structure of the physical space and the geometric properties of a set of cameras which provide pictures from which measurements can be made. We usually think of the physical space as being embedded in a three-dimensional euclidean space where measurements of lengths and angles do make sense. It turns out that for artificial systems, such as robots, this is not a mandatory viewpoint and that it is sometimes sufficient to think of the physical space as being embedded in an affine or even projective space. The question then arises of how to relate these models to image measurements and to geometric properties of sets of cameras. We show that in the case of two cameras, a stereo rig, the projective structure of the world can be recovered as soon as the epipolar geometry of the stereo rig is known and that this geometry is summarized by a single 3 3 matrix, which we called the fundamental matrix [1, 2]. The affine structure can then be recovered if we add to this information a projective transformation between the two images which is induced by the plane at infinity. Finally, the euclidean structure (up to a similitude) can be recovered if we add to these two elements the knowledge of two conics (one for each camera) which are the images of the absolute conic, a circle of radius p;1 in the plane at in nity. In all three cases we showhowthe three-dimensional information can be recovered directly from the images without explicitely reconstructing the scene structure. This defines a natural hierarchy of geometric structures, a set of three strata, that we overlay onthephysical world and which we show to be recoverable by simple procedures relying on two items, the physical space itself together with possibly, but not necessarily, some a priori information about it, and some voluntary motions of the set of cameras.

This paper gives a new method for image rectification, the process of resampling pairs of stereo images taken from widely differing viewpoints in order to produce a pair of "matched epipolar projections". These are projections in which the epipolar lines run parallel with the x-axis and consequently, disparities between the images are in the x-direction only. The method is based on an examination of the essential matrix of Longuet-Higgins which describes the epipolar geometry of the image pair. The approach taken is consistent with that recently advocated strongly by Faugeras ([1]) of av oiding camera calibration. The paper uses methods of projective geometry to define a matrix called the "epipolar transformation matrix" used to determine a pair of 2D projective transforms to be applied to the two images in order to match the epipolar lines. The advantages include the simplicity of the 2D projective transformation which allows very fast resampling as well as subsequent simplification in the identification of matched points and scene reconstruction.

There are three projective invariants of a set of six points in general position in space. It is well known that these invariants cannot be recovered from one image, however an invariant relationship does exist between space invariants and image invariants. This invariant relationship is first derived for a single image. Then this invariant relationship is used to derive the space invariants, when multiple images are available. This paper establishes that the minimum number of images for computing these invariants is three, and the computation of invariants of six points from three images can have as many as three solutions. Algorithms are presented for computing these invariants in closed form. The accuracy and stability with respect to image noise, selection of the triplets of images and distance between viewing positions are studied both through real and simulated images. Applications of these invariants are also presented. Both the results of Faugeras [1] and Hartley et al. [2] for...

. In this paper, problems related to depth, reconstruction and motion from a pair of projective images are studied under weak assumptions. Only relative information within each image is used, nothing about their interrelations or about camera calibration. Objects in the scene may be deformed between the imaging instants, provided that the deformations can be described locally by affine transformations. It is shown how the problems can be treated by a common method, based on a novel interpretation of a theorem in projective geometry of M. Chasles, and the notion of "affine shape". No epipolar geometry is used. The method also enables the computation of the "depth flow", i.e. a relative velocity in the direction of the ray of sight. Keywords: Depth, shape, reconstruction, motion, invariants. 1 Introduction Central problems in computer vision are concerned with reconstruction and recovery of motion from image pairs. A number of algorithms exist, in general based on iterative numerical t...

Relative Orientation is the recovery of the position and orientation of one imaging system relative to another from correspondences between five or more ray pairs. It is one of four core problems in photogrammetry and is of central importance in binocular stereo, as well as in long range motion vision. While five ray correspondences are sufficient to yield a finite number of solutions, more than five correspondences are used in practice to ensure an accurate solution using least squares methods. Most iterative schemes for minimizing the sum of squares of weighted errors require a good guess as a starting value. The author has previously published a method that finds the best solution without requiring an initial guess. In this paper an even simpler method is presented that utilizes the representation of rotations by unit quaternions. 1. See also: ``Relative Orientation,'' {\it International Journal of Computer Vision}, Vol.~4, No.~1, pp.~59--78, January 1990.

This paper provides a review on techniques for computing a three-dimensional model of a scene from a single moving camera, with unconstrained motion and unknown parameters. In the classical approach, called autocalibration or self-calibration, camera motion and parameters are recovered first, using rigidity; then structure is easily computed. Recently, new methods based on the idea of stratification have been proposed. They upgrade the projective structure, achievable from correspondences only, to the Euclidean structure, by exploiting all the available constraints.

This paper addresses the problem of recovering relative structure, in the form of an invariant, from two views of a 3D scene. The invariant structure is computed without any prior knowledge of camera geometry, or internal calibration, and with the property that perspective and orthographic projections are treated alike, namely, the system makes no assumption regarding the existence of perspective distortions in the input images. We show that

We present an algorithm that performs recursive estimation of ego-motion and ambient structure from a stream of monocular perspective images of a number of feature points. The algorithm is based on an Extended Kalman Filter (EKF) that integrates over time the instantaneous motion and structure measurements computed by a 2-perspective-views step. Key features of our filter are (1) global observability of the model, (2) complete on-line characterization of the uncertainty of the measurements provided by the two-views step. The filter is thus guaranteed to be well-behaved regardless of the particular motion undergone by the observer. Regions of motion space that do not allow recovery of structure (e.g. pure rotation) may be crossed while maintaining good estimates of structure and motion; whenever reliable measurements are available they are exploited. The algorithm works well for arbitrary motions with minimal smoothness assumptions and no ad hoc tuning. Simulations are presented that il...