This paper addresses the problem of camera selfcalibration, bundle adjustment and 3D reconstruction from line segments in two images of poorly-textured indoor scenes. First, we generate line segment correspondences, using an extended version of our previously proposed matching scheme. The first main contribution is a new method to identify polyhedral junctions resulting from the intersections of the line segments. At the same time, the images are segmented into planar polygons. This is done using an algorithm based on a Binary Space Partitioning (BSP) tree. The junctions are matched end points of the detected line segments and hence can be used to obtain the epipolar geometry. The essential matrix is considered for metric camera calibration. For better stability, the second main contribution consists in a bundle adjustment on the line segments and the camera parameters that reduces the number of unknowns by a maximum flow algorithm. Finally, a piecewise-planar 3D reconstruction is computed based on the segmentation of the BSP tree. The system’s performance is tested on some challenging examples. 1.

This paper presents a novel method for detection of image interest regions called Structure Guided Salient Regions (SGSR). Following the information theoretic route to saliency detection, we extend Kadir et al.’s Salient Region detector by exploiting image structure information. The detected SGSRs are highly distinct and very selective. For planar scenes, their performance on repeatability tests under viewpoint changes is comparable to the state of the art. For 3D scenes, SGSRs are more likely to be repeatably detected under viewpoint change. Their usefulness for wide baseline matching is demonstrated with a real-world example, where their comparative advantages are shown. 1

A framework is presented for estimating the pose of a camera based on images extracted from a single omnidirectional image of an urban scene, given a 2D map with building outlines with no 3D geometric information nor appearance data. The framework attempts to identify vertical corner edges of buildings in the query image, which we term VCLH, as well as the neighboring plane normals, through vanishing point analysis. A bottom-up process further groups VCLH into elemental planes and subsequently into 3D structural fragments modulo a similarity transformation. A geometric hashing lookup allows us to rapidly establish multiple candidate correspondences between the structural fragments and the 2D map building contours. A voting-based camera pose estimation method is then employed to recover the correspondences admitting a camera pose solution with high consensus. In a dataset that is even challenging for humans, the system returned a top-30 ranking for correct matches out of 3600 camera pose hypotheses (0.83 % selectivity) for 50.9 % of queries. 1.

Abstract—This paper presents a framework of layered graph matching for integrating graph partition and matching. The objective is to find an unknown number of corresponding graph structures in two images. We extract discriminative local primitives from both images and construct a candidacy graph whose vertices are matching candidates (i.e., a pair of primitives) and whose edges are either negative for mutual exclusion or positive for mutual consistence. Then we pose layered graph matching as a multicoloring problem on the candidacy graph and solve it using a composite cluster sampling algorithm. This algorithm assigns some vertices into a number of colors, each being a matched layer, and turns off all the remaining candidates. The algorithm iterates two steps: 1) Sampling the positive and negative edges probabilistically to form a composite cluster, which consists of a few mutually conflicting connected components (CCPs) in different colors and 2) assigning new colors to these CCPs with consistence and exclusion relations maintained, and the assignments are accepted by the Markov Chain Monte Carlo (MCMC) mechanism to preserve detailed balance. This framework demonstrates state-of-the-art performance on several applications, such as multi-object matching with large motion, shape matching and retrieval, and object localization in cluttered background. Index Terms—Graph matching, graph partitioning, DDMCMC, cluster sampling. Ç

Many vision tasks are posed as either graph partitioning (coloring) or graph matching (correspondence) problems. The former include segmentation and grouping, and the latter include wide baseline stereo, large motion, object tracking and recognition. In this paper, we present an integrated solution for both graph matching and graph partition using an effective sampling algorithm in a Bayesian framework. Given two images for matching, we extract two graphs using a primal sketch algorithm [4]. The graph nodes are linelets and primitives (junctions). Both graphs are automatically partitioned into an unknown number of K + 1 layers of subgraphs so that K pairs of subgraphs are matched and the remaining layer contains unmatched backgrounds. Each matched pair represent a ”moving object” with a TPS (Thin-Plate-Spline) transform to account for its deformations and a set of graph operators to edit the pair of subgraphs to achieve perfect structural match. The matching energy between two subgraphs includes geometric deformations, appearance dissimilarities, and the cost of graph editing operators. We demonstrate its application on two tasks: (i) large motion with occlusion, and (ii) automatic detection and recognition of common objects in a pair of images. 1.

Abstract — This paper presents an entire scheme for the estimation of a sparse disparity map from stereo pair images. In contrary to a dense disparity map, for which disparity values are calculated for each pixel of image, the sparse disparity map is determined only for some distinguished set of pixels from an image. The pixels belong to this set are called features. Therefore, in the first stage the algorithm for determining sparse disparity map has to be able to detect and specify some pixels as the feature pixels. In this article a method for specifying features is introduced. The core of the presented method relies on a fuzzy edges detector. The algorithm of fuzzy edges detection, as well as a manner enabling determination of the features ’ set, are introduced. The disparity is calculated at the fuzzy domain based on a similarity measure which is the correlation of fuzzy sets. The results of the obtained disparity maps for some benchmark stereo pairs, and the comparison with the well known Marr-Poggio-Grimson algorithm designed for sparse disparity map estimation are presented.

This thesis addresses the automatic calibration of two static surveillance cameras in a manmade world with orthogonal and parallel structures and a common ground plane. An approach is taken where the calibration of the interior orientation, the undistortion of the lens and the calibration of a camera’s rotation to the world perform before calibrating the camera centers, which allows methods that work in slightly overlapping as in non-overlapping views. We present a new incremental calibration composed of Expectation Maximization and Simulated Annealing that uses the uncertainties of noisy line segments to process a video stream instead of a single image. The advantage of video is that orthogonal and parallel edge

Abstract — To facilitate scene understanding and robot navigation in a modern urban area, we design a multilayer feature graph (MFG) based on two views from an on-board camera. The nodes of an MFG are features such as scale invariant feature transformation (SIFT) feature points, line segments, lines, and planes while edges of the MFG represent different geometric relationships such as adjacency, parallelism, collinearity, and coplanarity. MFG also connects the features in two views and the corresponding 3D coordinate system. Building on SIFT feature points and line segments, MFG is constructed using feature fusion which incrementally, iteratively, and extensively verifies the aforementioned geometric relationships using random sample consensus (RANSAC) framework. Physical experiments show that MFG can be successfully constructed in urban area and the construction method is demonstrated to be very robust in identifying feature correspondence. I.