We present a method for fusing two acquisition modes, 2D photographs and 3D LiDAR scans, for depth-layer decomposition of urban facades. The two modes have complementary characteristics: point cloud scans are coherent and inherently 3D, but are often sparse, noisy, and incomplete; photographs, on the other hand, are of high resolution, easy to acquire, and dense, but view-dependent and inherently 2D, lacking critical depth information. In this paper we use photographs to enhance the acquired LiDAR data. Our key observation is that with an initial registration of the 2D and 3D datasets we can decompose the input photographs into rectified depth layers. We decompose the input photographs into rectangular planar fragments and diffuse depth information from the corresponding 3D scan onto the fragments by solving a multi-label assignment problem. Our layer decomposition enables accurate repetition detection in each planar layer, using which we propagate geometry, remove outliers and enhance the 3D scan. Finally, the algorithm produces an enhanced, layered, textured model. We evaluate our algorithm on complex multi-planar building facades, where direct autocorrelation methods for repetition detection fail. We demonstrate how 2D photographs help improve the 3D scans by exploiting data redundancy, and transferring high level structural information to (plausibly) complete large missing regions. 1.

We introduce a novel framework for image-based 3D reconstruction of urban buildings based on symmetry priors. Starting from image-level edges, we generate a sparse and approximate set of consistent 3D lines. These lines are then used to simultaneously detect symmetric line arrangements while refining the estimated 3D model. Operating both on 2D image data and intermediate 3D feature representations, we perform iterative feature consolidation and effective outlier pruning, thus eliminating reconstruction artifacts arising from ambiguous or wrong stereo matches. We exploit non-local coherence of symmetric elements to generate precise model reconstructions, even in the presence of a significant amount of outlier image-edges arising from reflections, shadows, outlier objects, etc. We evaluate our algorithm on several challenging test scenarios, both synthetic and real. Beyond reconstruction, the extracted symmetry patterns are useful towards interactive and intuitive model manipulations.

Urban facades contain large-scale repetitions in the form of windows, doors, etc. Such elements often are in different configurations (e.g., open or closed) obscuring their regular arrangements to any direct low-level pixel matching based repetition detection. We propose a variation-factored representation for facade images by progressively favoring larger repeated structures while allowing relabeling using candidate element types. We formulate the problem as a Markov Random Field (MRF) based optimization, and evaluate the algorithm on a large number of benchmark facade images. Such a facade encoding is very compact and can be used for rapid generation of realistic 3D models with variations suitable for online map viewers or mobile navigation aids. 1.

The concept of symmetry has received significant attention in computer graphics and computer vision research in recent years. Numerous methods have been proposed to find and extract geometric symmetries and exploit such high-level structural information for a wide variety of geometry processing tasks. This report surveys and classifies recent developments in symmetry detection. We focus on elucidating the similarities and differences between existing methods to gain a better understanding of a fundamental problem in digital geometry processing and shape understanding in general. We discuss a variety of applications in computer graphics and geometry that benefit from symmetry information for more effective processing. An analysis of the strengths and limitations of existing algorithms highlights the plenitude of opportunities for future research both in terms of theory and applications. 1.

We propose a novel approach for the reconstruction of urban structures from 3D point clouds with an assumption of Manhattan World (MW) building geometry; i.e., the predominance of three mutually orthogonal directions in the scene. Our approach works in two-steps. First, the input points are classified according to the MW assumption into four local shape types: walls, edges, corners, and edge-corners. The classified points are organized into a connected set of clusters from which a volume description is extracted. The MW assumption allows us to robustly identify the fundamental shape types, describe the volumes within the bounding box, and reconstruct visible and occluded parts of the sampled structure. We show results of our reconstruction that has been applied to several synthetic and real-world 3D point datasets of various densities and from multiple viewpoints. Our method automatically reconstructs 3D building models from up to 10 million points in 10 to 60 seconds.

Laser scanners are often used to create 3D models of buildings for civil engineering applications. The current manual process is time-consuming and error-prone. This paper presents a method for using laser scanner data to model predominantly planar surfaces, such as walls, floors, and ceilings, despite the presence of significant amounts of clutter and occlusion, which occur frequently in natural indoor environments. Our goal is to recover the surface shape, detect and model any openings, and fill in the occluded regions. Our method identifies candidate surfaces for modeling, labels occluded surface regions, detects openings in each surface using supervised learning, and reconstructs the surface in the occluded regions. We evaluate the method on a large, highly cluttered data set of a building consisting of forty separate rooms.

Figure 1: Starting from an orthogonal input image (left top), the user interactively segments the façade into shapes (bottom left). Most split lines and symmetries are found automatically by the system, while the global façade structure is determined by the user. The input resolution is 1024 × 756, the number of visible shapes is 1346, and the modeling time is about 8 minutes (with material and depth assignments about 15 minutes). We propose a novel interactive framework for modeling building façades from images. Our method is based on the notion of coherence-based editing which allows exploiting partial symmetries across the façade at any level of detail. The proposed workflow mixes manual interaction with automatic splitting and grouping operations based on unsupervised cluster analysis. In contrast to previous work, our approach leads to detailed 3d geometric models with up to several thousand regions per façade. We compare our modeling scheme to others and evaluate our approach in a user study with an experienced user and several novice users.

IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder. For more details, see the IEEE Copyright Policy.