Prevailing efforts to study the standard formulation of motion and structure recovery have recently been focused on issues of sensitivity and robustness of existing techniques. While many cogent observations have been made and verified experimentally, many statements do not hold in general settings and make a comparison of existing techniques difficult. With an ultimate goal of clarifying these issues, we study the main aspects of motion and structure recovery: the choice of objective function, optimization techniques and sensitivity and robustness issues in the presence of noise. We clearly reveal the relationship among different objective functions, such as “(normalized) epipolar constraints,” “reprojection error” or “triangulation,” all of which can be unified in a new “optimal triangulation” procedure. Regardless of various choices of the objective function, the optimization problems all inherit the same unknown parameter space, the so-called “essential manifold.” Based on recent developments of optimization techniques on Riemannian manifolds, in particular on Stiefel or Grassmann manifolds, we propose a Riemannian Newton algorithm to solve the motion and structure recovery problem, making use of the natural differential geometric structure of the essential manifold. We provide a clear account of sensitivity and robustness of the proposed linear and nonlinear optimization techniques and study the analytical and practical equivalence of different objective functions. The geometric

For two–image structure from motion, we present a simple, exact expression for the least–squares image– reprojection error for finite motion that depends only on the motion. Optimal estimates of the structure and motion can be computed by minimizing this expression with respect to the motion parameters only. This gives a fast, reliable, and optimal algorithm. Also, we present a solution to the triangulation problem: an exact, explicit expression for the 3D structure given the motion. We identify a new ambiguity in recovering the structure for known motion. We use our exact expression for the least–squares error to study the error’s properties experimentally and demonstrate that it often has several local minima for forward or backward motion estimates. Most of these results assume that the camera is calibrated and use a version of the least–squares error that is most appropriate for a spherical imaging surface. We point out the advantages of the spherical error for standard cameras with planar imaging surfaces. We briefly discuss variants of our approach that apply to the standard least–squares error in the image plane and uncalibrated cameras. We present an improved version of the Sampson error, the standard first–order approximation to the least–squares error in the image plane, which gives better results experimentally.

We demonstrate the existence of a new approximate ambiguity in structure from motion which occurs as generically as the bas--relief ambiguity but applies more strongly to scenes with larger depth variation. It occurs for moderate translations and field of view (as for the bas--relief ambiguity) and applies to multi--frame, finite--motion sequences where the camera moves roughly along a line, as well as to optical flow. Previous work on the bas--relief ambiguity gave a partial characterization of the error sensitivities in recovering the camera heading, assuming that the scene was non--planar and that the heading was sufficiently different from the view direction. Our analysis completes the understanding of the error sensitivities under these conditions. 1 Introduction This paper demonstrates the existence of a new approximate ambiguity in structure from motion (SFM) which occurs as generically as the bas--relief ambiguity but differs from it in important ways---for instance, the new...

We present a novel algorithm for optimally segmenting dynamic scenes containing multiple rigidly moving objects. We cast the motion segmentation problem as a constrained nonlinear least squares problem which minimizes the reprojection error subject to all multibody epipolar constraints. By converting this constrained problem into an unconstrained one, we obtain an objective function that depends on the motion parameters only (fundamental matrices), but is independent on the segmentation of the image features. Therefore, our algorithm does not iterate between feature segmentation and single body motion estimation. Instead, it uses standard nonlinear optimization techniques to simultaneously recover all the fundamental matrices, without prior segmentation. We test our approach on a real sequence.

In this report we address a problem of Euclidean structure and motion recovery from image sequences and propose a linear method for determining the Euclidean motion and structure information up to a single universal scale regardless of the projection model. We formulate the problem in the "joint image space" and first review the existing multilinear constraints between m-images of n-points using exterior algebraic notation. It is well known that the projective constraints capture the information about the motion between individual frames and are used to recover it up to a scale. We show how the structural scale information which is lost during the projection process can be recovered using additional Euclidean constraints and propose a linear algorithm for obtaining compatible scales of the joint image matrix entries. We discuss further issues dealing with the uniqueness of the recovery and occlusion. The presented theory and algorithms are developed for both the discrete and differenti...

Abstract not found

Generalized Principal Component Analysis (GPCA): an Algebraic Geometric Approach to Subspace Clustering and Motion Segmentation by Ren e Esteban Vidal Doctor of Philosophy in Engineering -- Electrical Engineering and Computer Sciences University of California at Berkeley Professor Shankar Sastry, Chair Simultaneous data segmentation and model estimation refers to the problem of estimating a collection of models from sample data points, without knowing which points correspond to which model. This is a challenging problem in many disciplines, such as machine learning, computer vision, robotics and control, that is usually regarded as "chicken-and-egg". This is because if the segmentation of the data was known, one could easily fit a single model to each group of points. Conversely, if the models were known, one could easily find the data points that best fit each model. Since in practice neither the models nor the segmentation of the data are known, most of the existing approaches start with an initial estimate for the either the segmentation of the data or the model parameters and then iterate between data segmentation and model estimation. However, the convergence of iterative algorithms to the global optimum is in general very sensitive to initialization of both the number of models and the model parameters. Finding a good initialization remains a challenging problem.

Abstract not found

Abstract. The prevailing efforts to study the standard formulation of motion and structure recovery have been recently focused on issues of sensitivity and robustness of existing techniques. While many cogent observations have been made and verified experimentally, many statements do not hold in general settings and make a comparison of existing techniques difficult. With an ultimate goal of clarifying these issues we study the main aspects of the problem: the choice of objective functions, optimization techniques and the sensitivity and robustness issues in the presence of noise. We clearly reveal the relationship among different objective functions, such as “(normalized) epipolar constraints”, “reprojection error ” or “triangulation”, which can all be be unified in a new “ optimal triangulation” procedure formulated as a constrained optimization problem. Regardless of various choices of the objective function, the optimization problems all inherit the same unknown parameter space, the so called “essential manifold”, making the new optimization techniques on Riemanian manifolds directly applicable. Using these analytical results we provide a clear account of sensitivity and robustness of the proposed linear and nonlinear optimization techniques and study the analyticaland practicalequivalence of different objective functions. The geometric characterization of criticalpoints of a function defined on essential manifold and the simulation results clarify the difference between the effect of bas relief ambiguity and other types of local minima leading to a consistent interpretations of simulation results over large range of signal-to-noise ratio and variety of configurations. 1 1

We describe three approaches to 2-image and >= 2-image structure from motion. First, we present a new approximation to the least-squares image-reprojection error for 2 images. It depends only on the motion unknowns and is much more accurate than previous approximations such as the (weighted) coplanarity, especially for forward camera motions. We use this error to compute tight, rigorous upper and lower bounds on the true error and to study its properties experimentally. We demonstrate that the true error has many local minima for forward motions even when the motion is large. We propose and experimentally test a second approach, which is potentially more robust than bundle adjustment. Last, we describe algorithms for >= 2 images that reconstruct from the measured 2D affine deformations of image patches.