Abstract

Abstract. In this paper, we provide a principled explanation of how knowledge in global 3-D structural invariants, typically captured by a group action on a symmetric structure, can dramatically facilitate the task of reconstructing a 3-D scene from one or more images. More importantly, since every symmetric structure admits a “canonical” coordinate frame with respect to which the group action can be naturally represented, the canonical pose between the viewer and this canonical frame can be recovered too, which explains why symmetric objects (e.g., buildings) provide us overwhelming clues to their orientation and position. We give the necessary and sufficient conditions in terms of the symmetry (group) admitted by a structure under which this pose can be uniquely determined. We also characterize, when such conditions are not satisfied, to what extent this pose can be recovered. We show how algorithms from conventional multiple-view geometry, after properly modified and extended, can be directly applied to perform such recovery, from all “hidden images ” of one image of the symmetric structure. We also apply our results to a wide range of applications in computer vision and image processing such as camera self-calibration, image segmentation and global orientation, large baseline feature matching, image rendering and photo editing, as well as visual illusions (caused by symmetry if incorrectly assumed).

Citations

935	Mean shift: A robust approach toward feature space analysis - Comaniciu, Meer - 2002
120	Single view metrology - Criminisi, Reid, et al. - 2000
109	An Invitation to 3-D Vision: From Images to Geometric Models - Ma, Soatto, et al. - 2003
100	Recovering surface shape and orientation from texture - Witkin - 1981
70	ªComputing Local Surface Orientation and Shape from Texture for Curved Surfaces,º Int'l - Malik, Rosenholtz - 1997
65	Symmetry as a continuous feature - Zabrodsky, Peleg, et al. - 1995
57	Motion and Structure from Image Sequences - WENG, HUANG - 1993
50	3D symmetry detection using the extended Gaussian image - Sun, Sherrah - 1997
49	The Perception of the Visual - Gibson - 1950
48	Extracting projective structure from single perspective views of 3d point sets - ROTHWELL, FORSYTH, et al. - 1993
40	On the detection of axes of symmetry of symmetric and almost symmetric planner images - Marola - 1989
39	Shape from texture for smooth curved surfaces in perspective projection - G˚arding - 1992
27	Planar grouping for automatic detection of vanishing lines and points - Schaffalitzky, Zisserman
26	Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography - Fishler, Bolles - 1981
25	Symmetry - Weyl - 1952
22	3-D reconstruction using mirror images based on a plane symmetry recovering method - Mitsumoto, Tamura, et al. - 1992
21	Shape from texture and contour by weak isotropy - Garding - 1993
20	Y.: Detecting symmetry in grey level images: The global optimization approach - Kiryati, Gofman - 1996
19	Shape from symmetry—detecting and exploiting symmetry in affine images - Mukherjee, Zisserman, et al. - 1995
19	Using bilateral symmetry to improve 3d reconstruction from image sequences - Zabrodsky, Weinshall - 1997
17	R.: Geometric saliency of curve correspondances and grouping of symmetric comntours - Cham, Cipolla - 1996
16	On Perpendicular Texture: Why do we see more flowers in the distance - LEUNG, MALIK - 1997
16	ªSurface Orientation from Texture: Isotropy or Homogeneity �or Both)?,º - Rosenholtz, Malik - 1997
15	T.Poggio. Symmetric 3D Objects are an Easy Case for 2D Object Recognition - Vetter - 1995
12	Mirror and point symmetry under perspective skewing - Gool, Proesmans, et al. - 1996
12	Comments on symmetry as a continuous feature - Kanatani - 1997
10	Geometric segmentation of perspective images based on symmetry groups - Yang, Rao, et al. - 2003
9	The foundations of geometry and the non-euclidean plane, Intext Educational - Martin - 1972
7	Symmetry in perspective - Carlsson - 1998
6	Frieze and Wallpaper Symmetry Groups Classification under Affine and Perspective Distortion - Liu, Collins - 1998
5	Elements of the study of figures - Fedorov - 1953
5	Kruppa's Equations Revisited: its Degeneracy, Renormalization and Relations to Chirality - Ma, Vidal, et al. - 2000
4	Über die Bewegungsgruppen des n–dimensionalen euclidischen Raumes mit einem endlichen Fundamentalbereich - Bieberbach - 1910
4	Vision. W.H.Freeman and Company - MARR - 1982
3	Symmetry-based photo-editing - Huang, Hong, et al.
3	Detecting rotational symmetries using normalized convolution - Liebowitz, Zisserman - 1998
2	Large baseline matching and reconstruction from symmetry cells - Huang, Yang, et al. - 2004
1	Symmetry in the plane - Fedorov
1	1891b. Symmetry of finite figures - Fedorov
1	Symmetry of Crystals.Translated from the 1949 Russian Edition. American Crystallographic Association - Fedorov - 1971
1	Reconstruction of 3-D deformed symmetric curves from perspective images without discrete features - Hong, Ma, et al. - 2004
1	Characterizing the stability of 3D invariants derived from 3D translational symmetry - Liu, Walker, et al. - 1995
1	On Symmetry and Multiple-View Geometry 265 - Ma, Koˇsecká, et al. - 2002
1	Vision Science: Photons to Phenomenology - Plamer - 1999