## A New Multistage Approach to Motion and Structure Estimation: From Essential Parameters to Euclidean Motion Via Fundamental Matrix (1996)

Citations: | 20 - 1 self |

### BibTeX

@TECHREPORT{Zhang96anew,

author = {Zhengyou Zhang},

title = {A New Multistage Approach to Motion and Structure Estimation: From Essential Parameters to Euclidean Motion Via Fundamental Matrix},

institution = {},

year = {1996}

}

### OpenURL

### Abstract

The classical approach to motion and structure estimation problem from two perspective projections consists of two stages: (i) using the 8-point algorithm to estimate the 9 essential parameters defined up to a scale factor, which is a linear estimation problem; (ii) refining the motion estimation based on some statistically optimal criteria, which is a nonlinear estimation problem on a five-dimensional space. Unfortunately, the results obtained using this approach are often not satisfactory, especially when the motion is small or when the observed points are close to a degenerate surface (e.g. plane). The problem is that the second stage is very sensitive to the initial guess, and that it is very difficult to obtain a precise initial estimate from the first stage. This is because we perform a projection of a set of quantities which are estimated in a space of 8 dimensions, much higher than that of the real space which is five-dimensional. We propose in this paper a novel approach by introducing...

### Citations

1958 |
Matrix computations
- Golub, Loan
- 1996
(Show Context)
Citation Context ...singular value), and U and V are orthogonal matrices. Then, it can be shown that F = U SV T (21) with S = diag (s 1 ; s 2 ; 0) is the matrix of rank-2 which minimizes the Frobenius norm of M \Gamma F =-=[11]-=-. It is easy to verify that Fe e 1 = 0 and F T e e 2 = 0 : (22) Therefore, e e 1 = [e 11 ; e 12 ; e 13 ] T and e e 2 = [e 21 ; e 22 ; e 23 ] T are equal to the last column of V and U, respectively. Fr... |

1828 |
Robust Statistics
- Huber
- 1981
(Show Context)
Citation Context ...lace the squared residuals r 2 i by another functions of the residuals, yielding min X i ae(r i ) ; where ae is a symmetric, positive-denite function with a unique minimum at zero. For example, Huber =-=[16]-=- employed the squared error for small residuals and the absolute error for large residuals. The M-estimators can be implemented as a weighted least-squares problem. That is, we use ae(r i ) = w i r 2 ... |

1263 |
Three-dimensional computer vision, a geometric viewpoint
- Faugeras
- 1993
(Show Context)
Citation Context ...f E to 1. The second solves the classical problem: min ffl kUfflk 2 subject to kfflk = p 2 : (9) The constraint on the norm of ffl is derived from the fact that R is an orthonormal matrix and ktk = 1 =-=[6]-=-. The solution is the eigenvector of U T U associated with the smallest eigenvalue. This approach, known as the eight-point algorithm, was proposed by Longuet-Higgins [21] and has been extensively stu... |

1086 |
Robust regression and outlier detection
- Rousseeuw, Leroy
- 1987
(Show Context)
Citation Context ...ith that with p = 7. This is because we decrease the probability to have a good subsample when increasing the number of matches in each subsample. We think that p = 7 is a good trade-ooe. As noted in =-=[33]-=-, the LMedS eOEciency is poor in the presence of Gaussian noise. The eOEciency of a method is dened as the ratio between the lowest achievable variance for the estimated parameters and the actual vari... |

893 | Shape and motion from image streams under orthography: a factorization method
- Tomasi, Kanade
- 1992
(Show Context)
Citation Context ...to [42, 38] and Appendix B for a technique which uses the least-median-squares method to detect false matches. We also mention recent work on recovering motion and structure from long image sequences =-=[26, 4, 5, 44, 37, 30, 2, 36, 34, 20]-=-. The classical approach to motion and structure estimation problem from two given sets of matched image points consists of two stages: (i) using the 8-point algorithm to estimate the 9 essential para... |

719 | Closed-form solution of absolute orientation using unit quaternions
- HORN
- 1987
(Show Context)
Citation Context ... at most ten real solutions are possible in this case, but the algorithm is quite complex. When n ? 5, we usually have a unique solution, but in some special cases we may have at most three solutions =-=[13, 22, 27]-=-. The algorithm for n = 6 is complex, and is not addressed here. For n = 7, rank(U) = 7. Through singular value decomposition, we obtain vectors ffl 1 and ffl 2 which span the null space of U. The nul... |

630 |
A computer algorithm for reconstructing a scene from two projections
- Longuet-Higgins
- 1981
(Show Context)
Citation Context ...them between images; (ii) determine motion and structure from corresponding features. The earlier work was mainly on the development of linear algorithms and the existence and uniqueness of solutions =-=[21, 40, 9, 27]-=-. More recently, a number of researchers developed algorithms which are noise-resistant by using a suOEcient number of correspondences [8, 35, 41, 17]. Least-squares techniques are used to smooth out ... |

467 |
The interpretation of visual motion
- Ullman
- 1979
(Show Context)
Citation Context ...tect false matches. Instead of perspective views, the structure and motion problem for orthographic or, more generally, aOEne projections has also been extensively studied since Ullman's pioneer work =-=[47, 16, 22, 39, 37]-=-. We also mention recent work on recovering motion and structure from long image sequences [30, 5, 6, 51, 43, 34, 2, 42, 40, 24]. The classical approach to motion and structure estimation problem from... |

322 | Determining the epipolar geometry and its uncertainty: A review
- Zhang
- 1998
(Show Context)
Citation Context ...n is conducted over the above 7D parameter space, instead of the 5D motion space. The minimization is nonlinear, and we use the matrix estimated in (5) as the initial guess. The reader is referred to =-=[11]-=- for a detailed review of different techniques for estimating the fundamental matrix. Note that this intermediate stage can be applied to normalized image coordinates as well as pixel image coordinate... |

313 |
Uniqueness and estimation of three-dimensional motion parameters of rigid objects with curved surfaces
- Tsai, Huang
- 1984
(Show Context)
Citation Context ...them between images; (ii) determine motion and structure from corresponding features. The earlier work was mainly on the development of linear algorithms and the existence and uniqueness of solutions =-=[21, 40, 9, 27]-=-. More recently, a number of researchers developed algorithms which are noise-resistant by using a suOEcient number of correspondences [8, 35, 41, 17]. Least-squares techniques are used to smooth out ... |

292 |
Affine structure from motion
- Koenderink, Doorn
- 1991
(Show Context)
Citation Context ...tect false matches. Instead of perspective views, the structure and motion problem for orthographic or, more generally, aOEne projections has also been extensively studied since Ullman's pioneer work =-=[47, 16, 22, 39, 37]-=-. We also mention recent work on recovering motion and structure from long image sequences [30, 5, 6, 51, 43, 34, 2, 42, 40, 24]. The classical approach to motion and structure estimation problem from... |

252 |
The levenberg-marquardt algorithm: Implementation and theory
- Moré
- 1978
(Show Context)
Citation Context ...of perspective projection, the solution to the above problem demands the use of numerical nonlinear minimization technique such as the Levenberg-Marquardt algorithm implemented in the MINPACK library =-=[28]-=-. An initial guess on the motion and structure is required, which can be obtained by using the techniques described previously. The exact value of the covariance matrixsij is very diOEcult to obtain i... |

233 | The fundamental matrix: Theory, algorithms, and stability analysis
- Luong, Faugeras
- 1996
(Show Context)
Citation Context ... (2) where [t] \Theta is an antisymmetric matrix defined by t such that [t] \Theta x = t \Theta x for all 3-D vector x (\Theta denotes the cross product). Matrix F is knowns as the fundamental matrix =-=[6, 7]-=-. Equation (2) is a fundamental constraint underlying any two images if they are perspective projections of one and the same scene. For convenience, we use p to denote a point in the normalized image ... |

191 | Recovering 3D shape and motion from image streams using nonlinear least squares
- Szeliski, Kang
- 1994
(Show Context)
Citation Context ...to [42, 38] and Appendix B for a technique which uses the least-median-squares method to detect false matches. We also mention recent work on recovering motion and structure from long image sequences =-=[26, 4, 5, 44, 37, 30, 2, 36, 34, 20]-=-. The classical approach to motion and structure estimation problem from two given sets of matched image points consists of two stages: (i) using the 8-point algorithm to estimate the 9 essential para... |

157 |
Theory of reconstruction from image motion
- Maybank
- 1993
(Show Context)
Citation Context ...them between images; (ii) determine motion and structure from corresponding features. The earlier work was mainly on the development of linear algorithms and the existence and uniqueness of solutions =-=[21, 40, 9, 27]-=-. More recently, a number of researchers developed algorithms which are noise-resistant by using a suOEcient number of correspondences [8, 35, 41, 17]. Least-squares techniques are used to smooth out ... |

153 |
Motion and structure from feature correspondences: areview,”in
- Huang, Netravali
- 1994
(Show Context)
Citation Context ...terest in Computer Vision since its infancy, and is still an active domain of research. There are a large number of pieces of work reported in the literature in this domain. The reader is referred to =-=[29, 1, 14]-=- for a review. The problem is usually divided into two steps: (i) extract features (usually points or line) and match them between images; (ii) determine motion and structure from corresponding featur... |

147 |
On the computation of motion from sequences of images - A review
- Aggarwal, Nandhakumar
- 1988
(Show Context)
Citation Context ...terest in Computer Vision since its infancy, and is still an active domain of research. There are a large number of pieces of work reported in the literature in this domain. The reader is referred to =-=[29, 1, 14]-=- for a review. The problem is usually divided into two steps: (i) extract features (usually points or line) and match them between images; (ii) determine motion and structure from corresponding featur... |

142 |
The calibration problem for stereo
- Faugeras, Toscani
(Show Context)
Citation Context ...coordinate system to the camera coordinate system. The most general matrix A can be written as A = 2 4 ff u c u 0 0 ff v v 0 0 0 1 3 5 ; (1) where the ve parameters in A are known through calibration =-=[10, 39]-=-. RR n\Sigma2910 6 Zhengyou Zhang The rst and second images are respectively denoted by I 1 and I 2 . A point m in the image plane I i is noted as m i . The second subscript, if any, will indicate the... |

132 |
Motion from point matches: Multiplicity of solutions
- Faugeras, Maybank
- 1990
(Show Context)
Citation Context |

131 | In defense of the 8 point algorithm,” in
- Hartley
- 1997
(Show Context)
Citation Context ...te the essential parameters with 8-point algorithm (5). The obtained matrix is denoted by E 1 . Step 2: Estimate a rank-2 matrix, denoted by E 2 , from E 1 by setting the smallest singular value to 0 =-=[12, 11]-=-, and compute the seven parameters from E 2 . Step 3: Refine the seven parameters by minimizing the sum of squared distances between points and their epipolar lines, i.e., the objective function (6). ... |

130 | Optimal motion and structure estimation
- Weng, Ahuja, et al.
- 1993
(Show Context)
Citation Context ...hms and the existence and uniqueness of solutions [21, 40, 9, 27]. More recently, a number of researchers developed algorithms which are noise-resistant by using a suOEcient number of correspondences =-=[8, 35, 41, 17]-=-. Least-squares techniques are used to smooth out noise. In these works, the authors assume that matches are given and are correct. In real applications, however, among the feature correspondences est... |

108 |
Motion Segmentation and Outlier Detection
- Torr
- 1995
(Show Context)
Citation Context ...hes (called outliers in terms of robust statistics), sometimes even only one, will completely perturb the motion and structure estimation so that the result will be useless. The reader is referred to =-=[42, 38]-=- and Appendix B for a technique which uses the least-median-squares method to detect false matches. We also mention recent work on recovering motion and structure from long image sequences [26, 4, 5, ... |

96 | Recursive 3-D motion estimation from a monocular image sequence
- Broida, Chandrashekhar, et al.
- 1990
(Show Context)
Citation Context ...to [42, 38] and Appendix B for a technique which uses the least-median-squares method to detect false matches. We also mention recent work on recovering motion and structure from long image sequences =-=[26, 4, 5, 44, 37, 30, 2, 36, 34, 20]-=-. The classical approach to motion and structure estimation problem from two given sets of matched image points consists of two stages: (i) using the 8-point algorithm to estimate the 9 essential para... |

60 |
Some properties of the E matrix in two-view motion estimation
- Huang, Faugeras
- 1989
(Show Context)
Citation Context ... 1 e m 1 , and e p 2 = A \Gamma1 2 e m 2 . Let E = [t] \Theta R, which is known as the Essential matrix. It was introduced by Longuet-Higgins [21], and its property has been studied in the literature =-=[15, 9, 27]-=-. Now, we can write equation (4) as e p T 2 Ee p 1 = 0 : (5) Because the magnitude of t can never be recovered from two perspective images, we set ktk = 1. The relationship between E and F is readily ... |

57 |
Matrice Fondamentale et Calibration Visuelle sur l’Environment
- Luong
- 1992
(Show Context)
Citation Context ...ives equation (4). ffl Equation (4) can also be interpreted as the point m 2 lying on the epipolar line of m 1 . Let F = A \GammaT 2 [t] \Theta RA \Gamma1 1 ; which is known as the fundamental matrix =-=[7, 24]-=-. The epipolar line of m 1 , denoted by l m1 in Fig. 1, is the projection of the semi-line m 1 C 1 on the second image, and we have l m1 = F e m 1 (i.e. for all point m on line l m1 , we have e m T F ... |

56 |
Recursive Estimation of Structure and Motion using Relative Orientation
- Azarbayejani, Horowitz, et al.
- 1993
(Show Context)
Citation Context |

55 |
Motion and structure from orthographic projections
- Huang, Lee
(Show Context)
Citation Context ...tect false matches. Instead of perspective views, the structure and motion problem for orthographic or, more generally, aOEne projections has also been extensively studied since Ullman's pioneer work =-=[47, 16, 22, 39, 37]-=-. We also mention recent work on recovering motion and structure from long image sequences [30, 5, 6, 51, 43, 34, 2, 42, 40, 24]. The classical approach to motion and structure estimation problem from... |

51 |
A unified theory of structure from motion
- Spetsakis, Aloimonos
- 1990
(Show Context)
Citation Context ...hms and the existence and uniqueness of solutions [21, 40, 9, 27]. More recently, a number of researchers developed algorithms which are noise-resistant by using a suOEcient number of correspondences =-=[8, 35, 41, 17]-=-. Least-squares techniques are used to smooth out noise. In these works, the authors assume that matches are given and are correct. In real applications, however, among the feature correspondences est... |

49 | Stratification of 3-D vision: projective, affine, and metric representations
- Faugeras
(Show Context)
Citation Context ... (2) where [t] \Theta is an antisymmetric matrix defined by t such that [t] \Theta x = t \Theta x for all 3-D vector x (\Theta denotes the cross product). Matrix F is knowns as the fundamental matrix =-=[6, 7]-=-. Equation (2) is a fundamental constraint underlying any two images if they are perspective projections of one and the same scene. For convenience, we use p to denote a point in the normalized image ... |

39 |
3D Dynamic Scene Analysis,: A stereo based approach
- Zhang, Faugeras
- 1992
(Show Context)
Citation Context ...i k 2 subject to R T R = I and det(R) = 1 ; (12) where " i andsi are the i th row vectors of matrices E and [t] \Theta . This can be easily solved using the quaternion representation of 3-D rotat=-=ions [45]-=-. The ambiguity in the sign of E can now be resolved. We can use the procedure descri bed in Sect. 3.3 to reconstruct one 3-D point from image pairs. If the z coordinate of the reconstructed point is ... |

37 | Determination of ego-motion from matched points
- Harris
- 1987
(Show Context)
Citation Context ...ve developed an automatic and robust technique for matching two uncalibrated images. It consists of the following steps: ffl extract high curvature points from each image using Harris corner detector =-=[12]-=- or others; ffl nd match candidates for each high curvature point based on the normalized cross correlation; ffl disambiguate matches through fuzzy relaxation by using neighboring information; ffl det... |

30 |
The Reconstruction of a Scene from two Projections - Configurations that Defeat the 8-point Algorithm
- Longuet-Higgins
- 1984
(Show Context)
Citation Context ...igenvector of U T U associated with the smallest eigenvalue. This approach, known as the eight-point algorithm, was proposed by Longuet-Higgins [21] and has been extensively studied in the literature =-=[23, 40, 41, 19]-=-. It has been proven to be very sensitive to noise. Once we have estimated the essential matrix E, we can recover the motion (R; t). As E T t = 0, the relative location t is the solution of the follow... |

29 |
Motion and structure from point and line matches
- Faugeras, Lustman, et al.
- 1987
(Show Context)
Citation Context ...hms and the existence and uniqueness of solutions [21, 40, 9, 27]. More recently, a number of researchers developed algorithms which are noise-resistant by using a suOEcient number of correspondences =-=[8, 35, 41, 17]-=-. Least-squares techniques are used to smooth out noise. In these works, the authors assume that matches are given and are correct. In real applications, however, among the feature correspondences est... |

28 |
Incorporating motion error in multi-frame structure from motion
- Oliensis, Thomas
- 1991
(Show Context)
Citation Context |

26 |
Extended Structure and Motion Analysis from Monocular Image Sequences
- Cui, Weng, et al.
- 1990
(Show Context)
Citation Context |

23 | Motion and structure from two perspective views: from essential parameters to Euclidean motion through the fundamental matrix
- Zhang
- 1997
(Show Context)
Citation Context ...m yields much more reliable results than the standard one when the level of noise in data points is high or when data points are located close to a degenerate configuration. The reader is referred to =-=[13]-=- for more results including a set of real data with which the standard algorithm does not work while ours does. Fig. 1. Images of two planar grids hinged together with ` = 45 ffi . Gaussian noise of o... |

18 |
Epipolar Line Estimation
- Olsen
- 1992
(Show Context)
Citation Context ...ed the squared error for small residuals and the absolute error for large residuals. The M-estimators can be implemented as a weighted least-squares problem. That is, we use ae(r i ) = w i r 2 i . In =-=[31, 24]-=-, the following weight was used for the estimation of the epipolar geometry: w i = 8 ? ! ? : 1 jr i jsoe oe=jr i j oe ! jr i js3oe 0 3oe ! jr i j ; where oe is some estimated standard deviation of err... |

17 | Camera calibration, scene motion and structure recovery from point correspondences and fundamental matrices
- Luong, Faugeras
- 1997
(Show Context)
Citation Context ... parameters in a rank-2 matrix dened up to a scale factor (the scale factor and the rank-2 constraint remove two free parameters), and the fundamental matrix in the context of two uncalibrated images =-=[24, 25]-=- has exactly the same properties. There are several possible parameterizations for such a matrix, e.g. one can express one row (or column) of the fundamental matrix as the linear combination of the ot... |

16 |
Image sequences - ten (octal) years - from phenomenology towards a theoretical foundation
- Nagel
- 1986
(Show Context)
Citation Context ...terest in Computer Vision since its infancy, and is still an active domain of research. There are a large number of pieces of work reported in the literature in this domain. The reader is referred to =-=[29, 1, 14]-=- for a review. The problem is usually divided into two steps: (i) extract features (usually points or line) and match them between images; (ii) determine motion and structure from corresponding featur... |

16 | Motion estimation on the essential manifold
- SOATTO, FREZZA, et al.
- 1994
(Show Context)
Citation Context |

13 |
Multiframe image point matching and 3-d surface reconstruction
- Tsai
- 1983
(Show Context)
Citation Context ...coordinate system to the camera coordinate system. The most general matrix A can be written as A = 2 4 ff u c u 0 0 ff v v 0 0 0 1 3 5 ; (1) where the ve parameters in A are known through calibration =-=[10, 39]-=-. RR n\Sigma2910 6 Zhengyou Zhang The rst and second images are respectively denoted by I 1 and I 2 . A point m in the image plane I i is noted as m i . The second subscript, if any, will indicate the... |

11 |
Optimal visual motion estimation: a note
- Spetsakis, Aloimonos
- 1992
(Show Context)
Citation Context ...ms and the existence and uniqueness of solutions [25, 46, 11, 31]. More recently, a number of researchers developed algorithms which are noise-resistant by using a suOEcient number of correspondences =-=[3, 10, 41, 48, 20]-=-. Least-squares techniques are used to smooth out noise. In these works, the authors assume that matches are given and are correct. In real applications, however, among the feature correspondences est... |

10 |
Automatic singularity test for motion analysis by an information criterion
- Kanatani
- 1996
(Show Context)
Citation Context ... Images of two planar grids hinged together with ` = 45 ffi . Gaussian noise of oe = 0:5 has been added to each grid point 4.1 Computer Simulated Data We use the same conguration as that described in =-=[18]-=-. The object is composed of two planar grids which are hinged together with angles\Gamma `. When ` = 0, the object is planar, which is a degenerate conguration for the algorithms considered in this pa... |

6 |
Structure and motion from a sparse set of views
- Lee
- 1995
(Show Context)
Citation Context |

6 |
An Automatic and Robust Algorithm for Determining Motion and Structure from Two
- Zhang
- 1995
(Show Context)
Citation Context ...hes (called outliers in terms of robust statistics), sometimes even only one, will completely perturb the motion and structure estimation so that the result will be useless. The reader is referred to =-=[42, 38]-=- and Appendix B for a technique which uses the least-median-squares method to detect false matches. We also mention recent work on recovering motion and structure from long image sequences [26, 4, 5, ... |

5 |
Multiple interpretations of a pair of images of a
- Longuet-Higgins
- 1988
(Show Context)
Citation Context ... at most ten real solutions are possible in this case, but the algorithm is quite complex. When n ? 5, we usually have a unique solution, but in some special cases we may have at most three solutions =-=[13, 22, 27]-=-. The algorithm for n = 6 is complex, and is not addressed here. For n = 7, rank(U) = 7. Through singular value decomposition, we obtain vectors ffl 1 and ffl 2 which span the null space of U. The nul... |

4 |
Camera self-calibration: theory and experi ments
- Faugeras, Luong, et al.
- 1992
(Show Context)
Citation Context ...ives equation (4). ffl Equation (4) can also be interpreted as the point m 2 lying on the epipolar line of m 1 . Let F = A \GammaT 2 [t] \Theta RA \Gamma1 1 ; which is known as the fundamental matrix =-=[7, 24]-=-. The epipolar line of m 1 , denoted by l m1 in Fig. 1, is the projection of the semi-line m 1 C 1 on the second image, and we have l m1 = F e m 1 (i.e. for all point m on line l m1 , we have e m T F ... |

4 |
Strati��cation of 3-d vision: projective, aOEne, and metric representations
- Faugeras
- 1995
(Show Context)
Citation Context ...ives equation (4). ffl Equation (4) can also be interpreted as the point m 2 lying on the epipolar line of m 1 . Let F = A \GammaT 2 [t] \Theta RA \Gamma1 1 ; which is known as the fundamental matrix =-=[9, 28, 8]-=-. The epipolar line of m 1 , denoted by l m1 in Fig. 1, is the projection of the semi-line m 1 C 1 on the second image, and we have l m1 = F e m 1 (i.e. for all point m on line l m1 , we have e m T F ... |

4 |
Motion elds are hardly ever ambiguous
- Horn
- 1987
(Show Context)
Citation Context ... at most ten real solutions are possible in this case, but the algorithm is quite complex. When n ? 5, we usually have a unique solution, but in some special cases we may have at most three solutions =-=[15, 26, 31]-=-. The algorithm for n = 6 is complex, and is not addressed here. For n = 7, rank(U) = 7. Through singular value decomposition, we obtain vectors ffl 1 and ffl 2 which span the null space of U. The nul... |

3 |
Structure and motion from two orthographic views
- Pritt
- 1996
(Show Context)
Citation Context |