DMCA
Real-time Shading-based Refinement for Consumer Depth Cameras
| Citations: | 5 - 1 self |
BibTeX
@MISC{Wu_real-timeshading-based,
author = {Chenglei Wu and Matthias Nießner and Marc Stamminger and Shahram Izadi and Christian Theobalt},
title = {Real-time Shading-based Refinement for Consumer Depth Cameras},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Figure 1: Our method takes as input depth and aligned RGB images from any consumer depth camera (here a PrimeSense Carmine 1.09). Per-frame and in real-time we approximate the incident lighting and albedo, and use these for geometry refinement. From left: Example input depth and RGB image; raw depth input prior to refinement (rendered with normals and phong shading, respectively); our refined result, note detail on the eye (top right) compared to original depth map (bottom right); full 3D reconstruction using our refined depth maps in the real-time scan integration method of [Nießner et al. 2013] (far right) We present the first real-time method for refinement of depth data using shape-from-shading in general uncontrolled scenes. Per frame, our real-time algorithm takes raw noisy depth data and an aligned RGB image as input, and approximates the time-varying incident lighting, which is then used for geometry refinement. This leads to dramatically enhanced depth maps at 30Hz. Our algorithm makes few scene assumptions, handling arbitrary scene objects even under motion. To enable this type of real-time depth map enhancement, we contribute a new highly parallel algorithm that reformulates the inverse rendering optimization problem in prior work, allowing us to estimate lighting and shape in a temporally coherent way at video frame-rates. Our optimization problem is minimized using a new regular grid Gauss-Newton solver implemented fully on the GPU. We demonstrate results showing enhanced depth maps, which are comparable to offline methods but are computed orders of magnitude faster, as well as baseline comparisons with online filtering-based methods. We conclude with applications of our higher quality depth maps for improved real-time surface reconstruction and performance capture.
Keyphrases
consumer depth camera real-time shading-based refinement rgb image geometry refinement incident lighting optimization problem input depth performance capture nie ner et al refined depth map arbitrary scene object primesense carmine raw noisy depth data refined result computed order parallel algorithm inverse rendering optimization problem top right per frame original depth map coherent way enhanced depth map prior work real-time scan integration method raw depth input depth data magnitude faster real-time depth map enhancement baseline comparison first real-time method improved real-time surface reconstruction aligned rgb image note detail phong shading scene assumption video frame-rates new regular grid gauss-newton solver real-time algorithm depth map example input depth quality depth map time-varying incident lighting online filtering-based method bottom right general uncontrolled scene
