In this paper, we propose a high-order graph matching formulation to address non-rigid surface matching. The singleton terms capture the geometric and appearance similarities (e.g., curvature and texture) while the high-order terms model the intrinsic embedding energy. The novelty of this paper includes: 1) casting 3D surface registration into a graph matching problem that combines both geometric and appearance similarities and intrinsic embedding information, 2) the first implementation of high-order graph matching algorithm that solves a non-convex optimization problem, and 3) an efficient two-stage optimization approach to constrain the search space for dense surface registration. Our method is validated through a series of experiments demonstrating its accuracy and efficiency, notably in challenging cases of large and/or non-isometric deformations, or meshes that are partially occluded. 1.

Abstract—This paper presents a novel and efficient surface matching and visualization framework through the geodesic distanceweighted shape vector image diffusion. Based on conformal geometry, our approach can uniquely map a 3D surface to a canonical rectangular domain and encode the shape characteristics (e.g., mean curvatures and conformal factors) of the surface in the 2D domain to construct a geodesic distance-weighted shape vector image, where the distances between sampling pixels are not uniform but the actual geodesic distances on the manifold. Through the novel geodesic distance-weighted shape vector image diffusion presented in this paper, we can create a multiscale diffusion space, in which the cross-scale extrema can be detected as the robust geometric features for the matching and registration of surfaces. Therefore, statistical analysis and visualization of surface properties across subjects become readily available. The experiments on scanned surface models show that our method is very robust for feature extraction and surface matching even under noise and resolution change. We have also applied the framework on the real 3D human neocortical surfaces, and demonstrated the excellent performance of our approach in statistical analysis and integrated visualization of the multimodality volumetric data over the shape vector image. Index Terms—Surface Matching, Shape Vector Image, Multiscale Diffusion, Visualization. 1

Ricci flow is a powerful curvature flow method in geometric analysis. This work is the first application of surface Ricci flow in computer vision. We show that previous methods based on conformal geometries, such as harmonic maps and least-square conformal maps, which can only handle 3D shapes with simple topology are subsumed by our Ricci flow based method which can handle surfaces with arbitrary topology. Because the Ricci flow method is intrinsic and depends on the surface metric only, it is invariant to rigid motion, scaling, and isometric and conformal deformations. The solution to Ricci flow is unique and its computation is robust to noise. Our Ricci flow based method can convert all 3D problems into 2D domains and offers a general framework for 3D surface analysis. Large non-rigid deformations can be registered with feature constraints, hence we introduce a method that constrains Ricci flow computation using feature points and feature curves. Finally, we demonstrate the applicability of this intrinsic shape representation through standard shape analysis problems, such as 3D shape matching and registration. 1.

Automatic non-rigid registration of 3D time-varying data is fundamental in many vision and graphics applications such as facial expression analysis, synthesis, and recognition. Despite many research advances in recent years, it still remains to be technically challenging, especially for 3D dynamic, densely-sampled facial data with a large number of degrees of freedom (necessarily used to represent rich and subtle facial expressions). In this paper, we present a new method for automatic non-rigid registration of 3D dynamic facial data using least-squares conformal maps, and based on this registration method, we also develop a new framework of facial expression synthesis and transfer. Nowadays more and more 3D dynamic, densely-sampled data become prevalent with the advancement of novel 3D scanning techniques. To analyze and utilize such huge 3D data, an efficient non-rigid registration algorithm is needed to establish one-to-one inter-frame correspondences. Towards this goal, a non-rigid registration algorithm of 3D dynamic facial data is developed by using least-squares conformal maps with additional feature correspondences detected by employing active appearance models (AAM). The proposed method with additional, interior feature constraints guarantees that the non-rigid data will be accurately registered. The least-squares conformal maps between two 3D surfaces are globally optimized with the least angle distortion and the resulting 2D maps are stable and one-to-one. Furthermore, by using this non-rigid registration method, we develop a new system of facial expression synthesis and transfer. Finally, we perform a series of experiments to evaluate our non-rigid registration method and demonstrate its efficacy and efficiency in the applications of facial expression synthesis and transfer. 1. Introduction and Previous

3D surface matching is fundamental for shape analysis. As a powerful method in geometric analysis, Ricci flow can flexibly design metrics by prescribed target curvature. In this paper we describe a novel approach for matching surfaces with complicated topologies based on hyperbolic Ricci flow. For surfaces with negative Euler characteristics, such as a human face with holes (eye contours), the canonical hyperbolic metric is conformal to the original and can be efficiently computed. Then the surface can be canonically decomposed to hyperbolic hexagons. By matching the corresponding hyperbolic hexagons, the matching between surfaces can be easily established. Compared to existing methods, hyperbolic Ricci flow induces diffeomorphisms between surfaces with complicated topologies with negative Euler characteristics, while avoiding singularities. Furthermore, all the boundaries are intrinsically mapped to hyperbolic lines as alignment constraints. Finally, we demonstrate the applicability of this intrinsic shape representation for 3D face matching and registration. local isometric mapping [28], summation invariants [21], landmark-sliding [7], physics-based deformable models [30], Free-Form Deformation (FFD) [14], and Level-Set based methods [23]. However, many surface representations that use local geometric invariants can not guarantee a global convergence and might suffer from local minima in the presence of non-rigid deformations. To address this issue, many global parameterization methods have been developed recently based on conformal geometric maps

We address the person-independent recognition problem of facial expressions using static 3D face data. The novel approach to the facial expression recognition uses non-rigid registration of surface curvature features. 3D face data is cast onto 2D feature images, which are then subjected to elastic deformations in their parametric space. Each Action Unit (AU) detector is trained over its respective influence domain on the face. The registration task is incorporated in the multiresolution elastic deformation scheme, which yields adequate registration accuracy for mild pose variations. The algorithm is fully automatic and is free of the burden of first localizing anatomical facial points. The algorithm was tested on 22 facial action units of Facial Action Coding System. Promising results obtained indicate that we have an operative device for facial action unit detection, and an intermediate step to infer emotional or mental states. Moreover, experiments conducted with low intensity AU12- Lip Corner Puller points to the potential of 3D data and the proposed method in subtle expression detection. 1.

Non-rigid surface registration, particularly registration of human faces, finds a wide variety of applications in computer vision and graphics. We present a new automatic surface registration method which utilizes both attraction forces originating from geometrical and textural similarities, and stresses due to non-linear elasticity of the surfaces. Reference and target surfaces are first mapped onto their feature image planes, then these images are registered by subjecting them to local deformations, and finally 3D correspondences are established. Surfaces are assumed to be elastic sheets and are represented by triangular meshes. The internal elastic forces act as a regularizer in this ill-posed problem. Furthermore, the non-linear elasticity model allows us to handle large deformations, which can be essential, for instance, for facial expressions. The method has been tested successfully on 3D scanned human faces, with and without expressions. The algorithm runs quite efficiently using a multiresolution approach. 1.

Wide area surveillance situations require many sensors, thus making the use of highresolution cameras prohibitive because of high costs and exponential growth in storage. Small and low cost CCTV cameras may produce poor quality video, and high-resolution CCD cameras in wide area surveillance can still yield low-resolution images of the object of

Harmonic fields have been shown to provide effective guidance for a number of geometry processing problems. In this paper, we propose a method for fast updating of harmonic fields defined on polygonal meshes, enabling real-time insertion and deletion of constraints. Our approach utilizes the penalty method to enforce constraints in harmonic field computation. It maintains the symmetry of the Laplacian system and takes advantage of fast multi-rank updating and downdating of Cholesky factorization, achieving both speed and numerical stability. We demonstrate how the interactivity induced by fast harmonic field update can be utilized in several applications, including harmonic-guided quadrilateral remeshing, vector field design, interactive geometric detail modeling, and handle-driven shape editing and animation transfer with a dynamic handle set.

This paper presents a novel intrinsic 3D surface distance and its use in a complete probabilistic tracking framework for dynamic 3D data. Registering two frames of a deforming 3D shape relies on accurate correspondences between all points across the two frames. In the general case such correspondence search is computationally intractable. Common prior assumptions on the nature of the deformation such as near-rigidity, isometry or learning from a training set, reduce the search space but often at the price of loss of accuracy when it comes to deformations not in the prior assumptions. If we consider the set of all possible 3D surface matchings defined by specifying triplets of correspondences in the uniformization domain, then we introduce a new matching cost between two 3D surfaces. The lowest feature differences across this set of matchings that cause two points to correspond, become the matching cost of that particular correspondence. We show that for surface tracking applications, the matching cost can be efficiently computed in the uniformization domain. This matching cost is then combined with regularization terms that enforce spatial and temporal motion consistencies, into a maximum a posteriori (MAP) problem which we approximate using a Markov Random Field (MRF). Compared to previous 3D surface tracking approaches that either assume isometric deformations or consistent features, our method achieves dense, accurate tracking results, which we demonstrate through a series of dense, anisometric 3D surface tracking experiments. 1.