DMCA
Semantic structure from motion
| Venue: | In CVPR |
| Citations: | 43 - 9 self |
BibTeX
@INPROCEEDINGS{Bao_semanticstructure,
author = {Sid Yingze Bao and Silvio Savarese},
title = {Semantic structure from motion},
booktitle = {In CVPR},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Conventional rigid structure from motion (SFM) addresses the problem of recovering the camera parameters (motion) and the 3D locations (structure) of scene points, given observed 2D image feature points. In this paper, we propose a new formulation called Semantic Structure From Motion (SSFM). In addition to the geometrical constraints provided by SFM, SSFM takes advantage of both semantic and geometrical properties associated with objects in the scene (Fig. 1). These properties allow us to recover not only the structure and motion but also the 3D locations, poses, and categories of objects in the scene. We cast this problem as a max-likelihood problem where geometry (cameras, points, objects) and semantic information (object classes) are simultaneously estimated. The key intuition is that, in addition to image features, the measurements of objects across views provide additional geometrical constraints that relate cameras and scene parameters. These constraints make the geometry estimation process more robust and, in turn, make object detection more accurate. Our framework has the unique ability to: i) estimate camera poses only from object detections, ii) enhance camera pose estimation, compared to feature-point-based SFM algorithms, iii) improve object detections given multiple uncalibrated images, compared to independently detecting objects in single images. Extensive quantitative results on three datasets – LiDAR cars, street-view pedestrians, and Kinect office desktop – verify our theoretical claims. 1.
Keyphrases
semantic structure object detection geometrical property enhance camera pose estimation camera parameter theoretical claim datasets lidar car single image geometry estimation process street-view pedestrian new formulation extensive quantitative result kinect office desktop additional geometrical constraint image feature point feature-point-based sfm algorithm geometrical constraint object class scene point max-likelihood problem uncalibrated image scene parameter key intuition image feature unique ability estimate camera conventional rigid structure semantic information
