This paper describes the extraction of 3D geometrical data from image sequences, for the purpose of creating 3D models of objects in the world. The approach is uncalibrated - camera internal parameters and camera motion are not known or required. Processing an image sequence is underpinned by token correspondences between images. We utilise matching techniques which are both robust (detecting and discarding mismatches) and fully automatic. The matched tokens are used to compute 3D structure, which is initialised as it appears and then recursively updated over time. We describe a novel robust estimator of the trifocal tensor, based on a minimum number of token correspondences across an image triplet; and a novel tracking algorithm in which corners and line segments are matched over image triplets in an integrated framework. Experimental results are provided for a variety of scenes, including outdoor scenes taken with a hand-held camcorder. Quantitative statistics are included to asses...

This paper describes a theory and a practical algorithm for the autocalibration of a moving projective camera, from views of a planar scene. The unknown camera calibration, and (up to scale) the unknown scene geometry and camera motion are recovered from the hypothesis that the camera’s internal parameters remain constant during the motion. This work extends the various existing methods for non-planar autocalibration to a practically common situation in which it is not possible to bootstrap the calibration from an intermediate projective reconstruction. It also extends Hartley’s method for the internal calibration of a rotating camera, to allow camera translation and to provide 3D as well as calibration information. The basic constraint is that the projections of orthogonal direction vectors (points at infinity) in the plane must be orthogonal in the calibrated camera frame of each image. Abstractly, since the two circular points of the 3D plane (representing its Euclidean structure) lie on the 3D absolute conic, their projections into each image must lie on the absolute conic’s image (representing the camera calibration). The resulting numerical algorithm optimizes this constraint over all circular points and projective calibration parameters, using the inter-image homographies as a projective scene representation.

AbstractÐWe consider the problem of estimating parameters of a model described by an equation of special form. Specific models arise in the analysis of a wide class of computer vision problems, including conic fitting and estimation of the fundamental matrix. We assume that noisy data are accompanied by (known) covariance matrices characterizing the uncertainty of the measurements. A cost function is first obtained by considering a maximum-likelihood formulation and applying certain necessary approximations that render the problem tractable. A novel, Newton-like iterative scheme is then generated for determining a minimizer of the cost function. Unlike alternative approaches such as Sampson's method or the renormalization technique, the new scheme has as its theoretical limit the minimizer of the cost function. Furthermore, the scheme is simply expressed, efficient, and unsurpassed as a general technique in our testing. An important feature of the method is that it can serve as a basis for conducting theoretical comparison of various estimation approaches.

Abstract. The main result of this paper is a procedure for self-calibration of a moving camera from instantaneous optical ow. Under certain assumptions, this procedure allows the ego-motion and some intrinsic parameters of the camera to be determined solely from the instantaneous positions and velocities of a set of image features. The proposed method relies upon the use of a di erential epipolar equation that relates optical ow to the ego-motion and internal geometry of the camera. The paper presents a detailed derivation of this equation. This aspect of the work may be seen as a recasting into an analytical framework of the pivotal research ofVieville and Faugeras. 1 The information about the camera's ego-motion and internal geometry enters the di erential epipolar equation via two matrices. It emerges that the optical ow determines the composite ratio of some of the entries of the two matrices. It is shown that a camera with unknown focal length undergoing arbitrary motion can be self-calibrated via closed-form expressions in the composite ratio. The corresponding formulae specify ve ego-motion parameters, as well as the focal length and its derivative. An accompanying procedure is presented for reconstructing the viewed scene, up to scale, from the derived self-calibration data and the optical ow data. Experimental results are given to demonstrate the correctness of the approach. 1.

The aim of this paper is to explore a linear geometric algorithm for recovering the three dimensional motion of a moving camera from image velocities. Generic similarities and differences between the discrete approach and the differential approach are clearly revealed through a parallel development of an analogous motion estimation theory previously explored in [24, 26]. We present a precise characterization of the space of differential essential matrices, which gives rise to a novel eigenvalue-decomposition-based 3D velocity estimation algorithm from the optical flow measurements. This algorithm gives a unique solution to the motion estimation problem and serves as a differential counterpart of the well-known SVD-based 3D displacement estimation algorithm for the discrete case. Since the proposed algorithm only involves linear algebra techniques, it may be used to provide a fast initial guess for more sophisticated nonlinear algorithms [13]. Extensive simulation results are presented for evaluating the performance of our algorithm in terms of bias and sensitivity of the estimates with respect to di erent noise levels in image velocity measurements.

In this thesis several dioeerent cases of reconstruction of 3D objects from a number of 2D images, obtained by projective transformations, are considered. Firstly, the case where the images are taken by uncalibrated cameras, making it possible to reconstruct the object up to projective transformations, is described. The minimal cases of two images of seven points and three images of six points are solved, giving threefold solutions in both cases. Then linear methods for the cases where more points or more images are available are given, using multilinear constraints, based on a canonical representation of the multiple view geometry. The case of a continuous stream of images is also treated, giving multilinear constraints on the image coordinates and their derivatives. Secondly, the algebraic properties of the multilinear functions and the ideals generated by them are investigated. The main result is that the ideal generated by the bilinearities for three views have a primary decomposit...

We consider the problem of metrically reconstructing a scene viewed by a moving stereo head. The head comprises two cameras with coplanar optical axes arranged on a lateral rig, each camera being free to vary its angle of vergence. Under various constraints, we derive novel explicit forms for the epipolar equation, and show that a static stereo head constitutes a degenerate camera configuration for carrying out self-calibration. The situation is retrieved by consideration of a stereo head undergoing ground plane motion, and new closed-form solutions for self-calibration are derived. An error analysis reveals that reconstruction is adversely affected by inward-facing camera vergence angles that are similar in value, and by a principal point location whose horizontal component is in error. It is also shown that the adoption of domain-specific robust techniques for computation of the fundamental matrix can significantly improve the quality of scene reconstruction. Experiments conducted wi...

The aim of this paper is to explore intrinsic geometric methods of recovering the three dimensional motion of a moving camera from a sequence of images. Generic similarities between the discrete approach and the differential approach are revealed through a parallel development of their analogous motion estimation theories. We begin with a brief review of the (discrete) essential matrix approach, showing how to recover the 3D displacement from image correspondences. The space of normalized essential matrices is characterized geometrically: the unit tangent bundle of the rotation group is a double covering of the space of normalized essential matrices. This characterization naturally explains the geometry of the possible number of 3D displacements which can be obtained from the essential matrix. Second, a differential version of the essential matrix constraint previously explored by [19, 20] is introduced. We then present the precise characterization of the space of differential essentia...

We introduce a finite difference expansion for closely spaced cameras in projective vision, and use it to derive differential analogues of the finite-displacement projective matching tensors and constraints. The results are simpler, more general and easier to use than Astrom & Heyden's time-derivative based `continuous time matching constraints'. We suggest how to use the formalism for `tensor tracking' --- propagation of matching relations against a fixed base image along an image sequence. We relate this to nonlinear tensor estimators and show how `unwrapping the optimization loop' along the sequence allows simple `linear n point' update estimates to converge rapidly to statistically near-optimal, nearconsistent tensor estimates as the sequence proceeds. We also give guidelines as to when difference expansion is likely to be worthwhile as compared to a discrete approach. Keywords: Matching Constraints, Matching Tensors, Image Sequences, Tensor Tracking, Difference Expansion. 1 Intro...

The renormalisation technique of Kanatani is intended to iteratively minimise a cost function of a certain form while avoiding systematic bias inherent in the common method of minimisation due to Sampson. Within the computer vision community, the technique has generally proven difficult to absorb. This work presents an alternative derivation of the technique, and places it in the context of other approaches. We first show that the minimiser of the cost function must satisfy a special variational equation. A Newton-like, fundamental numerical scheme is presented with the property that its theoretical limit coincides with the minimiser. Standard statistical techniques are then employed to derive afresh several renormalisation schemes. The fundamental scheme proves pivotal in the rationalising of the renormalisation and other schemes, and enables us to show that the renormalisation schemes do not have as their theoretical limit the desired minimiser. The various minimisation schemes are finally subjected to a comparative performance analysis under controlled conditions.