## High resolution tracking of non-rigid 3D motion of densely sampled data using harmonic maps (2005)

### Cached

### Download Links

- [www.cs.sunysb.edu]
- [www.cs.sunysb.edu]
- [www.cs.sunysb.edu]
- [www.ri.cmu.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | In Proc. International Conference on Computer Vision |

Citations: | 20 - 8 self |

### BibTeX

@INPROCEEDINGS{Wang05highresolution,

author = {Yang Wang and Mohit Gupta and Song Zhang and Dimitris Samaras and Peisen Huang},

title = {High resolution tracking of non-rigid 3D motion of densely sampled data using harmonic maps},

booktitle = {In Proc. International Conference on Computer Vision},

year = {2005},

pages = {388--395}

}

### Years of Citing Articles

### OpenURL

### Abstract

We present a novel fully automatic method for high resolution, non-rigid dense 3D point tracking. High quality dense point clouds of non-rigid geometry moving at video speeds are acquired using a phase-shifting structured light ranging technique. To use such data for the temporal study of subtle motions such as those seen in facial expressions, an efficient non-rigid 3D motion tracking algorithm is needed to establish inter-frame correspondences. The novelty of this paper is the development of an algorithmic framework for 3D tracking that unifies tracking of intensity and geometric features, using harmonic maps with added feature correspondence constraints. While the previous uses of harmonic maps provided only global alignment, the proposed introduction of interior feature constraints guarantees that non-rigid deformations will be accurately tracked as well. The harmonic map between two topological disks is a diffeomorphism with minimal stretching energy and bounded angle distortion. The map is stable, insensitive to resolution changes and is robust to noise. Due to the strong implicit and explicit smoothness constraints imposed by the algorithm and the high-resolution data, the resulting registration/deformation field is smooth, continuous and gives dense one-to-one inter-frame correspondences. Our method is validated through a series of experiments demonstrating its accuracy and efficiency. 1. Introduction and Previous

### Citations

2222 |
D.: A method for registration of 3-d shapes
- BESL, MCKAY
- 1992
(Show Context)
Citation Context ...ed by the users [21]; the other calculates correspondences based on the geometry using a 3D deformable/morphable model [23, 29, 13, 36, 7, 5, 35, 9], or other 3D shape registration algorithms such as =-=[3, 39]-=-. In general, most of the existing methods rely on templates with relatively few degrees of freedom. While the recovered low dimensional configurations can often be used effectively in classification,... |

513 | STUETZLE W.: Multiresolution analysis of arbitrary meshes
- ECK, DEROSE, et al.
- 1995
(Show Context)
Citation Context ...t non-rigid 3D motion tracking algorithm is needed to establish inter-frame correspondences. In this paper, we propose such an algorithmic framework that uses a mathematical tool called harmonic maps =-=[27, 22, 10, 11]-=-. Harmonic maps were used in [37] to do surface matching, albeit focusing on rigid transformations. Given the source manifold M and the target manifold D, only the boundary condition u|∂M : ∂M → ∂D wa... |

483 | Iterative point matching for registration of free-form curves and surfaces
- Zhang
- 1994
(Show Context)
Citation Context ...ed by the users [21]; the other calculates correspondences based on the geometry using a 3D deformable/morphable model [23, 29, 13, 36, 7, 5, 35, 9], or other 3D shape registration algorithms such as =-=[3, 39]-=-. In general, most of the existing methods rely on templates with relatively few degrees of freedom. While the recovered low dimensional configurations can often be used effectively in classification,... |

388 | Face recognition based on fitting a 3D morphable model
- Blanz, Vetter
- 2003
(Show Context)
Citation Context ... attached to the object [15, 19, 2] or on feature correspondences manually selected by the users [21]; the other calculates correspondences based on the geometry using a 3D deformable/morphable model =-=[23, 29, 13, 36, 7, 5, 35, 9]-=-, or other 3D shape registration algorithms such as [3, 39]. In general, most of the existing methods rely on templates with relatively few degrees of freedom. While the recovered low dimensional conf... |

252 |
H.Sampson: Harmonic Mappings of Riemannian Manifolds
- unknown authors
- 1964
(Show Context)
Citation Context ...t non-rigid 3D motion tracking algorithm is needed to establish inter-frame correspondences. In this paper, we propose such an algorithmic framework that uses a mathematical tool called harmonic maps =-=[27, 22, 10, 11]-=-. Harmonic maps were used in [37] to do surface matching, albeit focusing on rigid transformations. Given the source manifold M and the target manifold D, only the boundary condition u|∂M : ∂M → ∂D wa... |

228 | Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion
- Black, Yacoob
- 1995
(Show Context)
Citation Context ...ression recognition, classification, detection of emotional states, etc. In the literature, most non-rigid object tracking and registration algorithms utilize image data from 2D video sequences, e.g. =-=[34, 4, 20, 1, 6, 31, 12, 13, 33, 30, 24]-=-. Previous methods establishing 3D inter-frame correspondences for non-rigid motion largely fall into two categories: One depends on markers attached to the object [15, 19, 2] or on feature correspond... |

198 | The space of human body shapes: reconstruction and parameterization from range scans
- Allen, Curless, et al.
- 2003
(Show Context)
Citation Context ...4, 20, 1, 6, 31, 12, 13, 33, 30, 24]. Previous methods establishing 3D inter-frame correspondences for non-rigid motion largely fall into two categories: One depends on markers attached to the object =-=[15, 19, 2]-=- or on feature correspondences manually selected by the users [21]; the other calculates correspondences based on the geometry using a 3D deformable/morphable model [23, 29, 13, 36, 7, 5, 35, 9], or o... |

179 |
Elementary differential geometry
- O’Neill
- 1997
(Show Context)
Citation Context ...t non-rigid 3D motion tracking algorithm is needed to establish inter-frame correspondences. In this paper, we propose such an algorithmic framework that uses a mathematical tool called harmonic maps =-=[27, 22, 10, 11]-=-. Harmonic maps were used in [37] to do surface matching, albeit focusing on rigid transformations. Given the source manifold M and the target manifold D, only the boundary condition u|∂M : ∂M → ∂D wa... |

175 |
The finite element method
- Hughes
- 1987
(Show Context)
Citation Context ...can assume that the local deformation between two adjacent frames is small. Furthermore, harmonic maps are easy to compute and robust to numerical errors. By using a traditional finite element method =-=[18]-=-, they are easy to implement. 3. The Non-Rigid Tracking Algorithm In this section, we present our novel fully automatic method for high resolution, non-rigid dense 3D point tracking using harmonic map... |

158 | Real-time 3D model acquisition
- RUSINKIEWICZ, HALL-HOLT, et al.
- 2002
(Show Context)
Citation Context ...al advances in digital imaging, digital projection display and personal computers have made real time 3D shape acquisition increasingly more feasible. Such ranging techniques include structured light =-=[26, 25]-=-, and space-time stereo [38, 8]. These systems can capture dense 3D data at a high frame rate. Recently, a high-resolution 3D expression data acquisition system was developed in [26] which captures hi... |

154 | Making faces
- Guenter, Grimm, et al.
- 1998
(Show Context)
Citation Context ...4, 20, 1, 6, 31, 12, 13, 33, 30, 24]. Previous methods establishing 3D inter-frame correspondences for non-rigid motion largely fall into two categories: One depends on markers attached to the object =-=[15, 19, 2]-=- or on feature correspondences manually selected by the users [21]; the other calculates correspondences based on the geometry using a 3D deformable/morphable model [23, 29, 13, 36, 7, 5, 35, 9], or o... |

141 | Spacetime faces: high resolution capture for modeling and animation - Zhang, Snavely, et al. |

126 | Tracking and modelling non-rigid objects with rank constraints
- Torresani, Yang, et al.
- 2001
(Show Context)
Citation Context ...ression recognition, classification, detection of emotional states, etc. In the literature, most non-rigid object tracking and registration algorithms utilize image data from 2D video sequences, e.g. =-=[34, 4, 20, 1, 6, 31, 12, 13, 33, 30, 24]-=-. Previous methods establishing 3D inter-frame correspondences for non-rigid motion largely fall into two categories: One depends on markers attached to the object [15, 19, 2] or on feature correspond... |

117 | Finding and tracking people from the bottom up
- Ramanan, Forsyth
- 2003
(Show Context)
Citation Context ...ression recognition, classification, detection of emotional states, etc. In the literature, most non-rigid object tracking and registration algorithms utilize image data from 2D video sequences, e.g. =-=[34, 4, 20, 1, 6, 31, 12, 13, 33, 30, 24]-=-. Previous methods establishing 3D inter-frame correspondences for non-rigid motion largely fall into two categories: One depends on markers attached to the object [15, 19, 2] or on feature correspond... |

116 | Real-time combined 2d+3d active appearance models
- Xiao, Baker, et al.
- 2004
(Show Context)
Citation Context ... attached to the object [15, 19, 2] or on feature correspondences manually selected by the users [21]; the other calculates correspondences based on the geometry using a 3D deformable/morphable model =-=[23, 29, 13, 36, 7, 5, 35, 9]-=-, or other 3D shape registration algorithms such as [3, 39]. In general, most of the existing methods rely on templates with relatively few degrees of freedom. While the recovered low dimensional conf... |

103 | U.: Expression cloning
- NOH, NEUMANN
(Show Context)
Citation Context ...ter-frame correspondences for non-rigid motion largely fall into two categories: One depends on markers attached to the object [15, 19, 2] or on feature correspondences manually selected by the users =-=[21]-=-; the other calculates correspondences based on the geometry using a 3D deformable/morphable model [23, 29, 13, 36, 7, 5, 35, 9], or other 3D shape registration algorithms such as [3, 39]. In general,... |

102 | Spacetime Stereo: A Unifying Framework for Depth from Triangulation
- Davis, Nehab, et al.
- 2005
(Show Context)
Citation Context ...digital projection display and personal computers have made real time 3D shape acquisition increasingly more feasible. Such ranging techniques include structured light [26, 25], and space-time stereo =-=[38, 8]-=-. These systems can capture dense 3D data at a high frame rate. Recently, a high-resolution 3D expression data acquisition system was developed in [26] which captures highly accurate geometry at speed... |

102 | Deformotion: Deforming motion, shape average and the joint registration and approximation of structure in images
- Yezzi, Soatto
- 2003
(Show Context)
Citation Context ... attached to the object [15, 19, 2] or on feature correspondences manually selected by the users [21]; the other calculates correspondences based on the geometry using a 3D deformable/morphable model =-=[23, 29, 13, 36, 7, 5, 35, 9]-=-, or other 3D shape registration algorithms such as [3, 39]. In general, most of the existing methods rely on templates with relatively few degrees of freedom. While the recovered low dimensional conf... |

82 | Resynthesizing facial animation through 3d model-based tracking
- PIGHIN, SZELISKI, et al.
- 1999
(Show Context)
Citation Context |

75 | Signal Matching Through Scale Space
- Terzopoulos, Kass
- 1986
(Show Context)
Citation Context |

65 |
Harmonic maps and their applications in surface matching
- Zhang, Hebert
- 1999
(Show Context)
Citation Context ...eeded to establish inter-frame correspondences. In this paper, we propose such an algorithmic framework that uses a mathematical tool called harmonic maps [27, 22, 10, 11]. Harmonic maps were used in =-=[37]-=- to do surface matching, albeit focusing on rigid transformations. Given the source manifold M and the target manifold D, only the boundary condition u|∂M : ∂M → ∂D was used to constrain and uniquely ... |

62 | 2d-shape analysis using conformal mapping
- MUMFORD
(Show Context)
Citation Context ...ly account for global rigid motion, but also subtle non-rigid deformations and hence achieve high accuracy registration and tracking. The theory of harmonic maps is based on conformal geometry theory =-=[14, 28]-=-; the harmonic map between two topological disks is a diffeomorphism with minimal stretching energy and bounded angle distortion. Harmonic maps are invariant for the same source surface with different... |

50 | Flexible flow for 3D nonrigid tracking and shape recovery
- Brand
- 2001
(Show Context)
Citation Context |

50 |
J.: Vision-based control of 3D facial animation
- J, XIAO, et al.
(Show Context)
Citation Context |

41 | resolution acquisition, learning and transfer of dynamic 3d facial expressions. Computer Graphics Forum 2004; 23(3): 677–686. Authors’ biographies: Yuru Pei received her Ph.D. in Computer Science from Peking University, China, in 2006, and M.S. in Compute
- Wang, Huang, et al.
- 1990
(Show Context)
Citation Context ...Initial data frame b) Initial tracked frame c) Data at the expression peak d) Tracked data at the peak e) Close-up at the peak ing algorithms have been proposed recently for 3D facial expression data =-=[38, 32]-=-. Tracking methods based on optical flow estimation[38, 13] can be sensitive to noise for textureless regions. A hierarchical tracking framework for high resolution 3D dynamic expression data was pres... |

35 |
Huang T S, 'Explanation-based facial motion tracking using a piecewise Bezier volume deformation model
- Tao
- 1999
(Show Context)
Citation Context |

33 |
A data-driven model for monocular face tracking
- Gokturk, Bouguet, et al.
(Show Context)
Citation Context |

28 |
3d tracking = classification + interpolation
- Tomasi, Petrov, et al.
- 2003
(Show Context)
Citation Context |

27 | Accurate Face Models from Uncalibrated and IllLit Video Sequences
- Dimitrijevic, Ilic, et al.
- 2004
(Show Context)
Citation Context |

27 | Establishing local correspondences towards compact representations of anatomical structures
- Huang, Paragios, et al.
- 2003
(Show Context)
Citation Context ...olve these problems, a generic face model (a coarser face mesh) is fitted to the first 3D scan frame in the initialization step, by a variational Free-Form Deformation (FFD) shape registration method =-=[32, 16]-=-. The FFD technique is employed only for fitting of the first frame, and not for subsequent tracking. 3.2. Global Alignment and Boundary Identification In the captured sequences, in addition to the no... |

27 | Subtly different facial expression recognition and expression intensity estimation
- Lien, Kanade, et al.
- 1998
(Show Context)
Citation Context |

24 |
Capturing subtle facial motions in 3D face tracking
- Wen, Huang
- 2003
(Show Context)
Citation Context |

21 | Statistical cue integration in dag deformable models
- Goldenstein, Vogler, et al.
(Show Context)
Citation Context |

17 | Surface classification using conformal structures
- Gu, Yau
(Show Context)
Citation Context ...ly account for global rigid motion, but also subtle non-rigid deformations and hence achieve high accuracy registration and tracking. The theory of harmonic maps is based on conformal geometry theory =-=[14, 28]-=-; the harmonic map between two topological disks is a diffeomorphism with minimal stretching energy and bounded angle distortion. Harmonic maps are invariant for the same source surface with different... |

17 | Face animation based on observed 3d speech dynamics
- Kalberer, Gool
(Show Context)
Citation Context ...4, 20, 1, 6, 31, 12, 13, 33, 30, 24]. Previous methods establishing 3D inter-frame correspondences for non-rigid motion largely fall into two categories: One depends on markers attached to the object =-=[15, 19, 2]-=- or on feature correspondences manually selected by the users [21]; the other calculates correspondences based on the geometry using a 3D deformable/morphable model [23, 29, 13, 36, 7, 5, 35, 9], or o... |

7 |
A hierarchical framework for high resolution facial expression tracking
- Huang, Zhang, et al.
- 2004
(Show Context)
Citation Context ...tler expression details that are generated by highly local skin deformations. We have achieved high accuracy tracking results on facial expression sequences, which are comparable to those reported in =-=[32, 17]-=-, using the same dense 3D data, while minimizing the amount of human labor required for preprocessing and initialization. The above mentioned level of accuracy, coupled with the automatic nature of ou... |

4 | Recovery and tracking of continuous 3d surfaces from stereo data using a deformable dual-mesh
- Akgul, Kambhamettu
- 1999
(Show Context)
Citation Context |

4 |
High resolution, real time 3d shape acquisition
- Zhang, Huang
- 2004
(Show Context)
Citation Context ...al advances in digital imaging, digital projection display and personal computers have made real time 3D shape acquisition increasingly more feasible. Such ranging techniques include structured light =-=[26, 25]-=-, and space-time stereo [38, 8]. These systems can capture dense 3D data at a high frame rate. Recently, a high-resolution 3D expression data acquisition system was developed in [26] which captures hi... |