No feature-based vision system can work unless good features can be identified and tracked from frame to frame. Although tracking itself is by and large a solved problem, selecting features that can be tracked well and correspond to physical points in the world is still hard. We propose a feature selection criterion that is optimal by construction because it is based on how the tracker works, and a feature monitoring method that can detect occlusions, disocclusions, and features that do not correspond to points in the world. These methods are based on a new tracking algorithm that extends previous Newton-Raphson style search methods to work under affine image transformations. We test performance with several simulations and experiments.

This paper is a survey of the theory and methods of photogrammetric bundle adjustment, aimed at potential implementors in the computer vision community. Bundle adjustment is the problem of refining a visual reconstruction to produce jointly optimal structure and viewing parameter estimates. Topics covered include: the choice of cost function and robustness; numerical optimization including sparse Newton methods, linearly convergent approximations, updating and recursive methods; gauge (datum) invariance; and quality control. The theory is developed for general robust cost functions rather than restricting attention to traditional nonlinear least squares.

Abstract: Before corresponding points in images taken with two cameras can be used to recover distances to objects in a scene, one has to determine the position and orientation of one camera relative to the other. This is the classic photogrammetric problem of relative orientation, central to the interpretation of binocular stereo information. Iterative methods for determining relative orientation were developed long ago; without them we would not have most of the topographic maps we do today. Relative orientation is also of importance in the recovery of motion and shape from an image sequence when successive frames are widely separated in time. Workers in motion vision are rediscovering some of the methods of photogrammetry. Described here is a simple iterative scheme for recovering relative orientation that, unlike existing methods, does not require a good initial guess for the baseline and the rotation. The data required is a pair of bundles of corresponding rays from the two projection centers to points in the scene. It is well known that at least five pairs of rays are needed. Less appears to be known about the existence of multiple solutions and their interpretation. These issues are discussed here. The unambiguous determination of all of the parameters of relative orientation is not possible when the observed points lie on a critical surface. These surfaces and their degenerate forms are analysed as well.

In this paper, we propose and analyze several methods to estimate a rigid transformation from a set of 3-D matched points or matched frames, which are important features in geometric algorithms. We also develop tools to predict and verify the accuracy of these estimations. The theoretical contributions are: an intrinsic model of noise for transformations based on composition rather than addition; a unified formalism for the estimation of both the rigid transformation and its covariance matrix for points or frames correspondences, and a statistical validation method to verify the error estimation, which applies even when no "ground truth" is available. We analyze and demonstrate on synthetic data that our scheme is well behaved. The practical contribution of the paper is the validation of our transformation estimation method in the case of 3-D medical images, which shows that an accuracy of the registration far below the size of a voxel can be achieved, and in the case of protein substructure matching, where frame features drastically improve both selectivity and complexity. 1.

The previous implementations of our Epipolar-Plane Image Analysis mapping technique demonstrated the feasibility and benefits of the approach, but were carried out for restricted camera geometries. The question of more general geometries made the technique's utility for autonomous navigation uncertain. We have developed a generalization of our analysis that (a) enables varying view direction, including variation over time (b) provides three-dimensional connectivity information for building coherent spatial descriptions of observed objects; and (c) operates sequentially, allowing initiation and refinement of scene feature estimates while the sensor is in motion. To implement this generalization it was necessary to develop an explicit description of the evolution of images over time. We have achieved this by building a process that creates a set of two-dimensional manifolds defined at the zeros of a three-dimensional spatiotemporal Laplacian. These manifolds represent explicitly both the spatial and temporal structure of the temporally evolving imagery, and we term them spatiotemporal surfaces. The surfaces are constructed incrementally, as the images are acquired. We describe a tracking mechanism that operates locally on these evolving surfaces in carrying out three-dimensional scene reconstruction.

The paper discusses objections against performance characterization of vision algorithms and explains their motivation. Short and long-term arguments are given which overcome these objections. The methodology for performance characterization is sketched to demonstrate the feasibility of empirical testing staffof vision algorithms. 0 Motivation For at least 10 years Computer Vision has been confronted with papers and discussions on the scientific value of its results and the difficulties in transferring the results to practical systems. A change of awareness seems to have happened: At the Computer Vision Workshop 1985 two controversial papers, with different views, agreed on the lack of theoretical research ([9], [13]), which should go along with the development of vision procedures: experimental proofs are not enough. The dialogue on 'Ignorance, Myopia, and Naivit'e in Computer Vision Systems' initiated by R. Jain and T. Binford ([10]) and the responses documented the necessity of ev...

. It is possible to recover the three-dimensional structure of a scene using images taken with uncalibrated cameras and pixel correspondences. But such a reconstruction can only be computed up to a projective transformation of the 3D space. Therefore, constraints have to be added to the reconstructed data in order to get the reconstruction in the euclidean space. Such constraints arise from knowledge of the scene: location of points, geometrical constraints on lines, etc. We first discuss here the type of constraints that have to be added then we show how they can be fed into a general framework. Experiments prove that the accuracy needed for industrial applications is reachable when measurements in the image have subpixel accuracy. Therefore, we show how a real camera can be mapped into an accurate projective camera and how accurate point detection improve the reconstruction results. 1 Introduction One of the principal goals of research in computer vision is to enable machines to per...

In this paper, we first review research results of camera self-calibration achieved in photogrammetry, robotics and computer vision. Then we propose a method for selfcalibration of robotic hand cameras by means of active motion. Through tracking a set of world points of unknown coordinates during robot motion, the internal parameters of the cameras (including distortions), the mounting parameters as well as the coordinates of the world points are estimated. The approach is fully autonomous, in that no initial guesses of the unknown parameters are to be provided from the outside by humans for the solution of a set of nonlinear equations. Sufficient conditions for a unique solution are derived in terms of controlled motion sequences. Methods to improve accuracy and robustness are proposed by means of best model identification and motion planning. Experimental results in both a simulated and a real environments are reported. Key words: self-calibration, hand-cameras, hand-eye calibration...

Abstract — This paper presents a consistent theory for describing indeterminacy and uncertainty of three-dimensional (3-D) reconstruction from a sequence of images. First, we give a group-theoretical analysis of gauges and gauge transformations. We then discuss how to evaluate the reliability of the solution that has indeterminacy and extend the Cramer-Rao lower bound to incorporate internal indeterminacy. We also introduce the free-gauge approach and define the normal form of a covariance matrix that is independent of particular gauges. Finally, we show simulated and real-image examples to illustrate the effect of gauge freedom on uncertainty description. Index Terms—Computer vision, Cramer-Rao lower bound, gauge transformation, geometric indeterminacy, statistical estimation, uncertainty description. I.

This paper presents a feature tracker for long image sequences based on simultaneously estimating the motions and deformations of a collection of adjacent image patches. By sharing common corner nodes, the patches achieve greater stability than independent patch trackers. Modeling full bilinear deformations enables tracking in sequences which have large non-translational motions and/or foreshortening effects. We demonstrate the advantages of our technique with respect to previous algorithms using experimental results. Keywords: motion analysis, multiframe feature tracking, affine patches c flDigital Equipment Corporation 1995. All rights reserved. 1 The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213-3890 Contents i Contents 1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 2 Previous work : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 3 Spline-based image registration : : : : : : : :...