In this paper we analyze in some detail the geometry of a pair of cameras, i.e. a stereo rig. Contrarily to what has been done in the past and is still done currently, for example in stereo or motion analysis, we do not assume that the intrinsic parameters of the cameras are known (coordinates of the principal points, pixels aspect ratio and focal lengths). This is important for two reasons. First, it is more realistic in applications where these parameters may vary according to the task (active vision). Second, the general case considered here, captures all the relevant information that is necessary for establishing correspondences between two pairs of images. This information is fundamentally projective and is hidden in a confusing manner in the commonly used formalism of the Essential matrix introduced by Longuet-Higgins [40]. This paper clarifies the projective nature of the correspondence problem in stereo and shows that the epipolar geometry can be summarized in one 3 \Theta 3 ma...

this article we focus on the study of the gestures of a person, but the same methodology could be applied to the study of robots motions or of other kinds of articulated objects. Some examples of applications are listed in the table 1.

Abstract. We present a method – termed Helmholtz stereopsis – for reconstructing the geometry of objects from a collection of images. Unlike most existing methods for surface reconstruction (e.g., stereo vision, structure from motion, photometric stereo), Helmholtz stereopsis makes no assumptions about the nature of the bidirectional reflectance distribution functions (BRDFs) of objects. This new method of multinocular stereopsis exploits Helmholtz reciprocity by choosing pairs of light source and camera positions that guarantee that the ratio of the emitted radiance to the incident irradiance is the same for corresponding points in the two images. The method provides direct estimates of both depth and field of surface normals, and consequently weds the advantages of both conventional and photometric stereopsis. Results from our implementations lend empirical support to our technique. 1

We address the structure-from-motion problem in the context of head modeling from video sequences for which calibration data is not available. This task is made challenging by the fact that correspondences are difficult to establish due to lack of texture and that a quasi-euclidean representation is required for realism.

We introduce a novel local image descriptor designed for dense wide-baseline matching purposes. We feed our descriptors to a graph-cuts based dense depth map estimation algorithm and this yields better wide-baseline performance than the commonly used correlation windows for which the size is hard to tune. As a result, unlike competing techniques that require many high-resolution images to produce good reconstructions, our descriptor can compute them from pairs of low-quality images such as the ones captured by video streams. Our descriptor is inspired from earlier ones such as SIFT and GLOH but can be computed much faster for our purposes. Unlike SURF which can also be computed efficiently at every pixel, it does not introduce artifacts that degrade the matching performance. Our approach was tested with ground truth laser scanned depth maps as well as on a wide variety of image pairs of different resolutions and we show that good reconstructions are achieved even with only two low quality images.

: Some medical interventions require knowing the correspondence between an MRI/CT image and the actual position of the patient. Examples occur in neurosurgery and radiotherapy, but also in video surgery (laparoscopy). We present in this paper three new techniques for performing this task without artificial markers. To do this, we find the 3D2D projective transformation (composition of a rigid displacement and a perspective projection) which maps a 3D object onto a 2D image of this object. Depending on the object model (curve or surface), and on the 2D image acquisition system (X-Ray, video), the techniques are different but the framework is common: ffl We first find an estimate of the transformation using bitangent lines or bitangent planes. These are first order semi-differential invariants [GMPO92]. ffl Then, introducing the normal or tangent, we define a distance between the 3D object and the 2D image, and we minimize it using extensions of the Iterative Closest Point algorithm (...

: We present a stratification of geometric information available from stereo in three levels: Euclidean, affine and projective, depending upon the kind of calibration that has been obtained for a stereo rig. We focus on the last two levels since they are mostly unexplored. We show how projective and affine calibration can be achieved from real images without the need of calibration patterns. We also show how to use this calibration to determine, for example, whether an obstacle is coming too close to the stereo rig or such useful information as the middle of a corridor or a road. Key-words: projective calibration, affine calibration, reconstruction, rectification (R'esum'e : tsvp) This work was partially supported by the EEC under Esprit Project 6448, Viva Unite de recherche INRIA Sophia-Antipolis 2004 route des Lucioles, BP 93, 06902 SOPHIA-ANTIPOLIS Cedex (France) Telephone : (33) 93 65 77 77 -- Telecopie : (33) 93 65 77 65 Applications de la vision non m'etrique `a certaines ...

We presentamultiview method for the computation of object shape and re ectance characteristics based on the integration of shape from shading (SFS) and stereo, for nonconstan talbedo and non-uniformly Lambertian surfaces. First we perform stereo tting on the input stereo pairs or image sequences. When the images are uncalibrated, w e recover the camera parameters using bundle adjustment. Based on the stereo result, we can automatically segment the albedo map (which istaken to be piece-wise constant) using a Minimum Description Length (MDL) based metric, to identify areas suitable for SFS (typically smooth textureless areas) and to deriv e illumination information. The shape and the illumination parameter estimates are re ned using a deformable model SFS algorithm, which iterates bet w een computing shape and illumination parameters. Our method takes into accoun tthe viewing angle dependent foreshortening and specularity e ects, and compensates as much as possible by utilizing information from more than one images. We demonstrate that we can extend the applicability of SFS algorithms to real world situations when some of its traditional assumptions are violated. We demonstrate our method by applying it to face shape reconstruction. Experimental results indicate a signi cant improvement over SFS-only or stereo-only based reconstruction. Model accuracy and detail are improved, especially in areas of low texture detail. Albedo information is retrieved and can be used to accurately re-render the model under di erent illumination conditions. 1

In this paper, we introduce a local image descriptor, DAISY, which is very efficient to compute densely. We also present an EM-based algorithm to compute dense depth and occlusion maps from wide-baseline image pairs using this descriptor. This yields much better results in wide-baseline situations than the pixel and correlation-based algorithms that are commonly used in narrowbaseline stereo. Also, using a descriptor makes our algorithm robust against many photometric and geometric transformations. Our descriptor is inspired from earlier ones such as SIFT and GLOH but can be computed much faster for our purposes. Unlike SURF, which can also be computed efficiently at every pixel, it does not introduce artifacts that degrade the matching performance when used densely. It is important to note that our approach is the first algorithm that attempts to estimate dense depth maps from wide-baseline image pairs, and we show that it is a good one at that with many experiments for depth estimation accuracy, occlusion detection, and comparing it against other descriptors on laser-scanned ground truth scenes. We also tested our approach on a variety of indoor and outdoor scenes with different photometric and geometric transformations and our experiments support our claim to being robust against these.

In this paper, we propose a face reconstruction technique that produces models that not only look good when texture mapped, but are also metrically accurate. Our method is designed to work with short uncalibrated video or movie sequences, even when the lighting is poor resulting in specularities and shadows that complicate the algorithm’s task. Our approach relies on optimizing the shape parameters of a sophisticated PCA based model given pairwise image correspondences as input. All that is required is enough relative motion between camera and subject so that we can derive structure from motion. By matching the results against laser scanning data, we will show that its precision is excellent and can be predicted as a function of the number and quality of the correspondences. This is important if one wishes to obtain the appropriate compromise between processing speed and quality of the results. Furthermore, our method is in fact not specific to faces and could equally be applied to any shape for which a shape model controlled with relatively small number of parameters exists. 1