Two images of a single scene/object are related by the epipolar geometry, which can be described by a 3×3 singular matrix called the essential matrix if images' internal parameters are known, or the fundamental matrix otherwise. It captures all geometric information contained in two images, and its determination is very important in many applications such as scene modeling and vehicle navigation. This paper gives an introduction to the epipolar geometry, and provides a complete review of the current techniques for estimating the fundamental matrix and its uncertainty. A well-founded measure is proposed to compare these techniques. Projective reconstruction is also reviewed. The software which we have developed for this review is available on the Internet.

In this paper we analyze in some detail the geometry of a pair of cameras, i.e. a stereo rig. Contrarily to what has been done in the past and is still done currently, for example in stereo or motion analysis, we do not assume that the intrinsic parameters of the cameras are known (coordinates of the principal points, pixels aspect ratio and focal lengths). This is important for two reasons. First, it is more realistic in applications where these parameters may vary according to the task (active vision). Second, the general case considered here, captures all the relevant information that is necessary for establishing correspondences between two pairs of images. This information is fundamentally projective and is hidden in a confusing manner in the commonly used formalism of the Essential matrix introduced by Longuet-Higgins [40]. This paper clarifies the projective nature of the correspondence problem in stereo and shows that the epipolar geometry can be summarized in one 3 \Theta 3 ma...

This work is in the context of motion and stereo analysis. It presents a new uni ed representation which will be useful when dealing with multiple views in the case of uncalibrated cameras. Several levels of information might be considered, depending on the availability of information. Among other things, an algebraic description of the epipolar geometry of N views is introduced, as well as a framework for camera self-calibration, calibration updating, and structure from motion in an image sequence taken by a camera which is zooming and moving at the same time. We show how a special decomposition of a set of two or three general projection matrices, called canonical enables us to build geometric descriptions for a system of cameras which are invariant with respect to a given group of transformations. These representations are minimal and capture completely the properties of each level of description considered: Euclidean (in the context of calibration, and in the context of structure from motion, which we distinguish clearly), a ne, and projective, that we also relate to each other. In the last case, a new decomposition of the well-known fundamental matrix is obtained. Dependencies, which appear when three or more views are available, are studied in the context of the canonic decomposition, and new composition formulas are established. The theory is illustrated by tutorial examples with real images.

We describe how 3D affine measurements may be computed from a single perspective view of a scene given only minimal geometric information determined from the image. This minimal information is typically the vanishing line of a reference plane, and a vanishing point for a direction not parallel to the plane. It is shown that affine scene structure may then be determined from the image, without knowledge of the camera's internal calibration (e.g. focal length), nor of the explicit relation between camera and world (pose). In particular, we show how to (i) compute the distance between planes parallel to the reference plane (up to a common scale factor); (ii) compute area and length ratios on any plane parallel to the reference plane; (iii) determine the camera's (viewer's) location. Simple geometric derivations are given for these results. We also develop an algebraic representation which unifies the three types of measurement and, amongst other advantages, permits a first order error pr...

: In this report, we address the problem of the prediction of new views of a given scene from existing weakly or fully calibrated views called reference views. Our method does not make use of a three-dimensional model of the scene, but of the existing relations between the images. The new views are represented in the reference views by a viewpoint and a retinal plane, i.e. by four points which can be chosen interactively. From this representation and from the constraints between the images, we derive an algorithm to predict the new views. We discuss the advantages of this method compared to the commonly used scheme : 3-D reconstruction-projection. We show some experimental results with synthetic and real data. Key-words: 3-D scene representation, multi-view stereo, image synthesis (R'esum'e : tsvp) This work was partially supported by DRET contract No 91-815/DRET/EAR and by the EEC under Esprit project 6448, Viva Unite de recherche INRIA Sophia-Antipolis 2004 route des Lucioles, BP 9...

It has been established that certain trilinear froms of three perspective views give rise to a tensor of 27 intrinsic coefficients [11]. We show in this paper that a permutation of the the trilinear coefficients produces three homography matrices (projective transformations of planes) of three distinct intrinsic planes, respectively. This, in turn, yields the result that 3D invariants are recovered directly --- simply by appropriate arrangement of the tensor's coefficients. On a secondary level, we show new relations between fundamental matrix, epipoles, Euclidean structure and the trilinear tensor. On the practical side, the new results extend the existing envelope of methods of 3D recovery from 2D views --- for example, new linear methods that cut through the epipolar geometry, and new methods for computing epipolar geometry using redundancy available across many views. 1 Introduction Given that three-dimensional (3D) objects in the world are modeled by point sets, then their proje...

We address the problem of reconstructing 3D space in a projective framework from two or more views, and the problem of artificially generating novel views of the scene from two given views (re-projection). We describe an invariance relation which provides a new description of structure, we call projective depth, which is captured by a single equation relating image point correspondences across two or more views and the homographies of two arbitrary virtual planes. The framework is based on knowledge of correspondence of features across views, is linear, extremely simple, and the computations of structure readily extends to over-determination using multiple views. Experimental results demonstrate a high degree of accuracy in both tasks - reconstruction and re-projection. Keywords---Visual Recognition, 3D Reconstruction from 2D Views, Projective Geometry, Algebraic and Geometric Invariants. I. Introduction The geometric relation between objects (or scenes) in the world and their imag...

We propose an affine framework for perspective views, captured by a single extremely simple equation based on a viewer-centered invariant we call relative affine structure. Via a number of corollaries of our main results we show that our framework unifies previous work -- including Euclidean, projective and affine -- in a natural and simple way, and introduces new, extremely simple, algorithms for the tasks of reconstruction from multiple views, recognition by alignment, and certain image coding applications.

We propose an affine framework for perspective views, captured by a single extremely simple equation based on a viewer-centered invariant we call relative affine structure. Via a number of corollaries of our main results we show that our framework unifies previous work --- including Euclidean, projective and affine --- in a natural and simple way. Finally, the main results were applied to a real image sequence for purpose of 3D reconstruction from 2D views. 1 Introduction The introduction of affine and projective tools into the field of computer vision have brought increased activity in the fields of structure from motion and recognition by alignment in the recent few years. The emerging realization is that non-metric information, although weaker than the information provided by depth maps and rigid camera geometries, is nonetheless useful in the sense that the framework may provide simpler algorithms, camera calibration is not required, more freedom in picture-taking is allowed --- ...

This paper considers projective reconstruction with a hierarchical computational structure of trifocal tensors that integrates feature tracking and geometrical validation of the feature tracks. The algorithm was embedded into a system aimed at completely automatic Euclidean reconstruction from uncalibrated handheld amateur video sequences. The algorithm was tested as part of this system on a number of sequences grabbed directly from a low-end video camera without editing. The proposed approach can be considered a generalisation of a scheme of [Fitzgibbon and Zisserman, ECCV ‘98]. The proposed scheme tries to adapt itself to the motion and frame rate in the sequence by finding good triplets of views from which accurate and unique trifocal tensors can be calculated. This is in contrast to the assumption that three consecutive views in the video sequence are a good choice. Using trifocal tensors with a wider span suppresses error accumulation and makes the scheme less reliant on bundle adjustment. The proposed computational structure may also be used with fundamental matrices as the basic building block. 1