Results 1 - 10
of
31
Performance Capture of Interacting Characters with Handheld
"... Abstract. We present an algorithm for marker-less performance capture of interacting humans using only three hand-held Kinect cameras. Our method reconstructs human skeletal poses, deforming surface geometry and camera poses for every time step of the depth video. Skeletal configurations and camera ..."
Abstract
-
Cited by 13 (6 self)
- Add to MetaCart
(Show Context)
Abstract. We present an algorithm for marker-less performance capture of interacting humans using only three hand-held Kinect cameras. Our method reconstructs human skeletal poses, deforming surface geometry and camera poses for every time step of the depth video. Skeletal configurations and camera poses are found by solving a joint energy minimization problem which optimizes the alignment of RGBZ data from all cameras, as well as the alignment of human shape templates to the Kinect data. The energy function is based on a combination of geometric correspondence finding, implicit scene segmentation, and correspondence finding using image features. Only the combination of geometric and photometric correspondences and the integration of human pose and camera pose estimation enables reliable performance capture with only three sensors. As opposed to previous performance capture methods, our algorithm succeeds on general uncontrolled indoor scenes with potentially dynamic background, and it succeeds even if the cameras are moving. 1
Coherent Spatiotemporal Filtering, Upsampling and Rendering of RGBZ Videos
- COMPUTER GRAPHICS FORUM (PROCEEDINGS OF EUROGRAPHICS)
, 2012
"... Sophisticated video processing effects require both image and geometry information. We explore the possibility to augment a video camera with a recent infrared time-of-flight depth camera, to capture high-resolution RGB and low-resolution, noisy depth at video frame rates. To turn such a setup into ..."
Abstract
-
Cited by 13 (3 self)
- Add to MetaCart
(Show Context)
Sophisticated video processing effects require both image and geometry information. We explore the possibility to augment a video camera with a recent infrared time-of-flight depth camera, to capture high-resolution RGB and low-resolution, noisy depth at video frame rates. To turn such a setup into a practical RGBZ video camera, we develop efficient data filtering techniques that are tailored to the noise characteristics of IR depth cameras. We first remove typical artefacts in the RGBZ data and then apply an efficient spatiotemporal denoising and upsampling scheme. This allows us to record temporally coherent RGBZ videos at interactive frame rates and to use them to render a variety of effects in unprecedented quality. We show effects such as video relighting, geometry-based abstraction and stylisation, background segmentation and rendering in stereoscopic 3D.
Real-time 3d reconstruction in dynamic scenes using point-based fusion
- IN: 3DV
, 2013
"... Real-time or online 3D reconstruction has wide appli-cability and receives further interest due to availability of consumer depth cameras. Typical approaches use a mov-ing sensor to accumulate depth measurements into a single model which is continuously refined. Designing such systems is an intricat ..."
Abstract
-
Cited by 11 (1 self)
- Add to MetaCart
Real-time or online 3D reconstruction has wide appli-cability and receives further interest due to availability of consumer depth cameras. Typical approaches use a mov-ing sensor to accumulate depth measurements into a single model which is continuously refined. Designing such systems is an intricate balance between reconstruction quality, speed, spatial scale, and scene assumptions. Existing online meth-ods either trade scale to achieve higher quality reconstruc-tions of small objects/scenes. Or handle larger scenes by trading real-time performance and/or quality, or by limiting the bounds of the active reconstruction. Additionally, many systems assume a static scene, and cannot robustly handle scene motion or reconstructions that evolve to reflect scene changes. We address these limitations with a new system for real-time dense reconstruction with equivalent quality to existing online methods, but with support for additional spatial scale and robustness in dynamic scenes. Our system is designed around a simple and flat point-based represen-tation, which directly works with the input acquired from range/depth sensors, without the overhead of converting be-tween representations. The use of points enables speed and memory efficiency, directly leveraging the standard graphics pipeline for all central operations; i.e., camera pose estima-tion, data association, outlier removal, fusion of depth maps into a single denoised model, and detection and update of dynamic objects. We conclude with qualitative and quantita-tive results that highlight robust tracking and high quality reconstructions of a diverse set of scenes at varying scales.
J.R.: Real-time head and hand tracking based on 2.5d data
- IEEE Trans. on Multimedia
, 2012
"... A novel real-time algorithm for head and hand tracking is proposed in this paper. This approach is based on 2.5D data from a range cam-era, which is exploited to resolve ambiguities and overlaps. Exper-imental results show high robustness against partial occlusions and fast movements. The estimated ..."
Abstract
-
Cited by 8 (2 self)
- Add to MetaCart
(Show Context)
A novel real-time algorithm for head and hand tracking is proposed in this paper. This approach is based on 2.5D data from a range cam-era, which is exploited to resolve ambiguities and overlaps. Exper-imental results show high robustness against partial occlusions and fast movements. The estimated positions are fairly stable, allowing the extraction of accurate trajectories which may be used for gesture classification purposes. Index Terms — head tracking, hand tracking, time-of-flight camera, gesture recognition, body tracking 1.
C.: Depth recovery using an adaptive color-guided auto-regressive model
- In ECCV
, 2012
"... Abstract. This paper proposes an adaptive color-guided auto-regressive (AR) model for high quality depth recovery from low quality measure-ments captured by depth cameras. We formulate the depth recovery task into a minimization of AR prediction errors subject to measurement con-sistency. The AR pre ..."
Abstract
-
Cited by 7 (1 self)
- Add to MetaCart
(Show Context)
Abstract. This paper proposes an adaptive color-guided auto-regressive (AR) model for high quality depth recovery from low quality measure-ments captured by depth cameras. We formulate the depth recovery task into a minimization of AR prediction errors subject to measurement con-sistency. The AR predictor for each pixel is constructed according to both the local correlation in the initial depth map and the nonlocal similarity in the accompanied high quality color image. Experimental results show that our method outperforms existing state-of-the-art schemes, and is versatile for both mainstream depth sensors: ToF camera and Kinect.
Computational Cameras: Convergence of Optics and Processing
, 2011
"... A computational camera uses a combination of optics and processing to produce images that cannot be captured with traditional cameras. In the last decade, computational imaging has emerged as a vibrant field of research. A wide variety of computational cameras has been demonstrated to encode more u ..."
Abstract
-
Cited by 7 (0 self)
- Add to MetaCart
A computational camera uses a combination of optics and processing to produce images that cannot be captured with traditional cameras. In the last decade, computational imaging has emerged as a vibrant field of research. A wide variety of computational cameras has been demonstrated to encode more useful visual information in the captured images, as compared with conventional cameras. In this paper, we survey computational cameras from two perspectives. First, we present a taxonomy of computational camera designs according to the coding approaches, including object side coding, pupil plane coding, sensor side coding, illumination coding, camera arrays and clusters, and unconventional imaging systems. Second, we use the abstract notion of light field representation as a general tool to describe computational camera designs, where each camera can be formulated as a projection of a high-dimensional light field to a 2-D image sensor. We show how individual optical devices transform light fields and use these transforms to illustrate how different computational camera designs (collections of optical devices) capture and encode useful visual information.
Projective Alignment of Range and Parallax Data
- In Proc. Computer Vision and Pattern Recognition
, 2011
"... HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte p ..."
Abstract
-
Cited by 5 (2 self)
- Add to MetaCart
(Show Context)
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et a ̀ la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Stereo Time-of-Flight
"... This paper describes a novel method to acquire depth images using a pair of ToF (Time of Flight) cameras. As opposed to approaches that filter, calibrate or do 3D reconstructions posterior to the image acquisition, we propose to combine the measurements of the two cameras at the acquisition level. T ..."
Abstract
-
Cited by 4 (0 self)
- Add to MetaCart
(Show Context)
This paper describes a novel method to acquire depth images using a pair of ToF (Time of Flight) cameras. As opposed to approaches that filter, calibrate or do 3D reconstructions posterior to the image acquisition, we propose to combine the measurements of the two cameras at the acquisition level. To do so, we define a three-stages procedure, during which we actively modify the infrared lighting of the scene: first, the two cameras emit an infrared signal one after the other (stages 1 and 2), and then, simultaneously (stage 3). Assuming the scene is static during the three stages, we gather the depth measurements obtained with both cameras and define a cost function to optimize the two depth images. A quantitative evaluation of the performance of the proposed method for different objects and stereo configurations is provided based on a simulation of the ToF cameras. Results on real images are also presented. Both in simulation and real images the stereo-ToF acquisition produces more accurate depth measurements. 1.
1 Using Time-of-Flight Measurements for Privacy-Preserving Tracking in a Smart Room
"... Abstract — We present a method for real-time person tracking and coarse pose recognition in a smart room using timeof-flight measurements. The time-of-flight images are severely downsampled to preserve the privacy of the occupants and simulate future applications that use single-pixel sensors in “sm ..."
Abstract
-
Cited by 3 (2 self)
- Add to MetaCart
(Show Context)
Abstract — We present a method for real-time person tracking and coarse pose recognition in a smart room using timeof-flight measurements. The time-of-flight images are severely downsampled to preserve the privacy of the occupants and simulate future applications that use single-pixel sensors in “smart ” ceiling panels. The tracking algorithms use grayscale morphological image reconstruction to avoid false detections, and are designed not to mistakenly detect pieces of furniture as people. A maximum likelihood estimation method using a simple Markov model was implemented for robust pose classification. We show that the algorithms work effectively even when the sensors are spaced apart by 25cm, using both real-world experiments and environmental simulation. Index Terms — time-of-flight, smart room, pose recognition, visual tracking, privacy preservation, occupancy detection I.
Computational Plenoptic Imaging
, 2011
"... The plenoptic function is a ray-based model for light that includes the color spectrum as well as spatial, temporal, and directional variation. Although digital light sensors have greatly evolved in the last years, one fundamental limitation remains: all standard CCD and CMOS sensors integrate over ..."
Abstract
-
Cited by 3 (0 self)
- Add to MetaCart
The plenoptic function is a ray-based model for light that includes the color spectrum as well as spatial, temporal, and directional variation. Although digital light sensors have greatly evolved in the last years, one fundamental limitation remains: all standard CCD and CMOS sensors integrate over the dimensions of the plenoptic function as they convert photons into electrons; in the process, all visual information is irreversibly lost, except for a two-dimensional, spatially-varying subset — the common photograph. In this state of the art report, we review approaches that optically encode the dimensions of the plenoptic function transcending those captured by traditional photography and reconstruct the recorded information computationally.
