This survey focuses on recognition performed by matching models of the three-dimensional shape of the face, either alone or in combination with matching corresponding two-dimensional intensity images. Research trends to date are summarized, and challenges confronting the development of more accurate three-dimensional face recognition are identified. These challenges include the need for better sensors, improved recognition algorithms, and more rigorous experimental methodology.

www.elsevier.com/locate/cviu A survey of approaches and challenges in

Abstract—We present a fully automatic face recognition algorithm and demonstrate its performance on the FRGC v2.0 data. Our algorithm is multimodal (2D and 3D) and performs hybrid (feature based and holistic) matching in order to achieve efficiency and robustness to facial expressions. The pose of a 3D face along with its texture is automatically corrected using a novel approach based on a single automatically detected point and the Hotelling transform. A novel 3D Spherical Face Representation (SFR) is used in conjunction with the Scale-Invariant Feature Transform (SIFT) descriptor to form a rejection classifier, which quickly eliminates a large number of candidate faces at an early stage for efficient recognition in case of large galleries. The remaining faces are then verified using a novel region-based matching approach, which is robust to facial expressions. This approach automatically segments the eyesforehead and the nose regions, which are relatively less sensitive to expressions and matches them separately using a modified Iterative Closest Point (ICP) algorithm. The results of all the matching engines are fused at the metric level to achieve higher accuracy. We use the FRGC benchmark to compare our results to other algorithms that used the same database. Our multimodal hybrid algorithm performed better than others by achieving 99.74 percent and 98.31 percent verification rates at a 0.001 false acceptance rate (FAR) and identification rates of 99.02 percent and 95.37 percent for probes with a neutral and a nonneutral expression, respectively. Index Terms—Biometrics, face recognition, rejection classifier, 3D shape representation. 1

In 3D face recognition systems, 3D facial shape information plays an important role. Various shape representations have been proposed in the literature. The most popular techniques are based on point clouds, surface normals, facial profiles, and statistical analysis of depth images. The contribution of the presented work can be divided into two parts: In the first part, we have developed face classifiers which use these popular techniques. A comprehensive comparison of these representation methods are given using 3D RMA dataset. Experimental results show that the linear discriminant analysis-based representation of depth images and point cloud representation perform best. In the second part of the paper, two different multiple-classifier architectures are developed to fuse individual shape-based face recognizers in parallel and hierarchical fashions at the decision level. It is shown that a significant performance improvement is possible when using rank-based decision fusion in ensemble methods.

Abstract. Many researches in face recognition have been dealing with the challenge of the great variability in head pose, lighting intensity and direction,facial expression, and aging. The main purpose of this overview is to describe the recent 3D face recognition algorithms. The last few years more and more 2D face recognition algorithms are improved and tested on less than perfect images. However, 3D models hold more information of the face, like surface information, that can be used for face recognition or subject discrimination. Another major advantage is that 3D face recognition is pose invariant. A disadvantage of most presented 3D face recognition methods is that they still treat the human face as a rigid object. This means that the methods aren’t capable of handling facial expressions. Although 2D face recognition still seems to outperform the 3D face recognition methods, it is expected that this will change in the near future. 1

Abstract—In this paper, we present an extensive study of 3-D face recognition algorithms and examine the benefits of various score-, rank-, and decision-level fusion rules. We investigate face recognizers from two perspectives: the data representation techniques used and the feature extraction algorithms that match best each representation type. We also consider novel applications of various feature extraction techniques such as discrete Fourier transform, discrete cosine transform, nonnegative matrix factorization, and principal curvature directions to the shape modality. We discuss and compare various classifier combination methods such as fixed rules and voting- and rank-based fusion schemes. We also present a dynamic confidence estimation algorithm to boost fusion performance. In identification experiments performed on FRGC v1.0 and FRGC v2.0 face databases, we have tried to find the answers to the following questions: 1) the relative importance of the face representation techniques vis-à-vis the types of features extracted; 2) the impact of the gallery size; 3) the conditions, under which subspace methods are preferable, and the compression factor; 4) the most advantageous fusion level and fusion methods; 5) the role of confidence votes in improving fusion and the style of selecting experts in the fusion; and 6) the consistency of the conclusions across different databases. Index Terms—Classifier selection, face representation, feature extraction, fusion, independent component analysis (ICA), nonnegative matrix factorization (NMF), 3-D face recognition. I.

The face is an easily collectible and non-intrusive biometric used for the authentication and identification of individuals. 2D face recognition techniques are sensitive to changes in illumination, makeup and pose. We present a fully automatic 3D face recognition algorithm that overcomes these limitations. During the enrollment, 3D faces in the gallery are represented by third order tensors which are indexed by a 4D hash table. During online recognition, tensors are computed for a probe and are used to cast votes to the tensors in the gallery using the hash table. Gallery faces are ranked according to their votes and a similarity measure based on a linear correlation coefficient and registration error is calculated only for the high ranked faces. The face with the highest similarity is declared as the recognized face. Experiments were performed on a database of 277 subjects and a rank one recognition rate of 86.4% was achieved. Our results also show that our algorithm’s execution time is insensitive to the gallery size. 1.

Face recognition (FR) is the preferred mode of identity recognition by humans: It is natural, robust and unintrusive. However, automatic FR techniques have failed to match up to expectations: Variations in pose, illumination and expression limit the performance of 2D FR techniques. In recent years, 3D FR has shown promise to overcome these challanges. With the availability of cheaper acquisition methods, 3D face recognition can be a way out of these problems, both as a stand-alone method, or as a supplement to 2D face recognition. We review the relevant work on 3D face recognition here, and discuss merits of different representations and recognition algorithms.

In this paper, we review and compare 3D face registration and recognition algorithms which are based solely on 3D shape information and analyze methods based on the fusion of shape features. We have analyzed two different registration algorithms which produce a dense correspondence between faces. The first algorithm non-linearly warps faces to obtain registration, while the second algorithm allows only rigid transformations. Registration is handled with the use of an average face model which significantly fastens the registration process. As 3D facial features, we compare the use of 3D point coordinates, surface normals, curvature-based descriptors, 2D depth images, and facial profile curves. Except for surface normals, these feature descriptors are frequently used in state-of-the-art 3D face recognizers. We also perform an in-depth analysis of decision-level fusion techniques such as fixedrules, voting schemes, rank-based combination rules, and novel serial fusion architectures. The results of the recognition and authentication experiments conducted on the 3D_RMA database indicate that: (i) in terms of face registration method, registration of faces without warping preserves more discriminatory information, (ii) in terms of 3D facial features, surface normals attain the best recognition performance, and (iii) fusion schemes such as product rules, improved consensus voting and proposed serial fusion schemes improve the classification accuracy. Experimental results on the 3D_RMA confirm these findings by obtaining %0.1 misclassification rate in recognition experiments, and %8.06 equal error rate in authentication experiments using surface normal-based features. It is also possible to improve the classification accuracy by %2.38 using fixed fusion rules when moderate-level classifiers are used. Key words: 3D face recognition, face registration, 3D surface descriptors

Abstract. Machine recognition of faces is very challenging because it is an interclass recognition problem and the variation in faces is very low compared to other biometrics. Global features have been extensively used for face recognition however they are sensitive to variations caused by expressions, illumination, pose, occlusions and makeup. We present a novel 3D local feature for automatic face recognition which is robust to these variations. The 3D features are extracted by uniformly sampling local regions of the face in locally defined coordinate bases which makes them invariant to pose. The high descriptiveness of this feature makes it ideal for the challenging task of interclass recognition. In the 2D domain, we use the SIFT descriptor and fuse the results with the 3D approach at the score level. Experiments were performed using the FRGC v2.0 data and the achieved verification rates at 0.001 FAR were 98.5 % and 86.0% for faces with neutral and non-neutral expressions respectively. 1