A face recognition system must recognize a face from a novel image despite the variations between images of the same face. A common approach to overcoming image variations because of changes in the illumination conditions is to use image representations that are relatively insensitive to these variations. Examples of such representations are edge maps, image intensity derivatives, and images convolved with 2D Gabor-like filters. Here we present an empirical study that evaluates the sensitivity of these representations to changes in illumination, as well as viewpoint and facial expression. Our findings indicated that none of the representations considered is sufficient by itself to overcome image variations because of a change in the direction of illumination. Similar results were obtained for changes due to viewpoint and expression. Image representations that emphasized the horizontal features were found to be less sensitive to changes in the direction of illumination. However, systems...

This paper explores the representation of the human face by features based on shape and curvature of the face surface. Curvature captures many features necessary to accurately describe the face, such as the shape of the forehead, jawline, and cheeks, which are not easily detected from standard intensity images. Moreover, the value of curvature at a point on the surface is also viewpointinvariant. Until recently range data of high enough resolution and accuracy to perform useful curvature calculations on the scale of the human face had been unavailable. Although several researchers have worked on the problem of interpreting range data from curved (although usually highly geometrically structured) surfaces, the main approaches have centered on segmentation by signs of mean and Gaussian curvature which have not proved su cient for classi cation of human faces. This paper details the calculation of principal curvature for our particular data set, the calculation of general surface descriptors based on curvature, and the calculation of face speci c descriptors based both on curvature features and aprioriknowledge about the structure of the face. These face speci c descriptors can be incorporated into many di erent recognition strategies. We describe a system which implements one such strategy, depth template comparison, giving excellent recognition rates in our test cases. 1

An image of a face depends not only on its shape, but also on the viewpoint, illumination conditions, and facial expression. A face recognition system must overcome the changes in face appearance induced by these factors. This paper investigate two related questions: the capacity of the human visual system to generalize the recognition of faces to novel images, and the level at which this generalization occurs. We approach this problems by comparing the identification and generalization capacity for upright and inverted faces. For upright faces, we found remarkably good generalization to novel conditions. For inverted faces, the generalization to novel views was significantly worse for both new illumination and viewpoint, although the performance on the training images was similar to the upright condition. Our results indicate that at least some of the processes that support generalization across viewpoint and illumination are neither universal (because subjects did not generalize as e...

Principal component analysis (PCA) of face images is here related to subjects'performance on the same images. In two experiments subjects were shown a set of faces and asked to rate them for distinctiveness. They were subsequently shown a superset of faces and asked to identify those which appeared originally. Replicating previous work, we found that hits and false positives (FPs) did not correlate: those faces easy to identify as being "seen" were unrelated to those faces easy to reject as being "unseen". PCA was performed on three data sets: (i) face images with eye-position standardised; (ii) face images morphed to a standard template to remove shape information; (iii) the shape information from faces only. Analyses based upon PCA of shape-free faces gave high predictions of FPs, while shape information itself contributed only to hits. Furthermore, while FPs were generally predictable from components early in the PCA, hits appear to be accounted for by later components. We conclude that shape and "texture" (the image-based information remaining after morphing) may be used separately by the human face processing system, and that PCA of images offers a useful tool for understanding this system.

this paper and Jim Anderson for support. This paper has been written during a visiting professorship in Brown University made possible by a Fullbright scholarship (1986--1988). Correspondence about this paper should be adressed to: Herv'e Abdi, The University of Texas at Dallas, Program in Cognition, MS:GR4.1., Richardson, TX75083-0688, USA. email: herve@utdallas.edu .

A system for automatic face recognition is presented. It consists of several steps; Automatic detection of the eyes and mouth is followed by a spatial normalization of the images. The classification of the normalized images is carried out by a hybrid (supervised and unsupervised) Neural Network. Two methods for reducing the overfitting -- a common problem in high dimensional classification schemes -- are presented, and the superiority of their combination is demonstrated. Key words: Face recognition, Neural Networks, Interest points, Symmetry operator. To appear: Pattern Recognition Letters 17 (1996) 67-76 1 Introduction Automatic face recognition has gained much attention in recent years, due to the variety of potential applications, and the increase in computational power which enables effective implementation of algorithms. Traditionally, face recognition was based on extracting certain features (e.g. spatial location of facial features and their geometrical relations) [4, 20]....

Individual faces vary considerably in both the quality and quantity of the information they contain for recognition and for viewpoint generalization. In the present study, we assessed the typicality, recognizability, and viewpoint generalizability of individual faces using data from both human observers and from a computational model of face recognition across viewpoint change. The two-stage computational model incorporated a viewpoint alignment operation and a recognition-by-interpolation operation. An interesting aspect of this particular model is that the effects of typicality it predicts at the alignment and recognition stages dissociate, such that face typicality is beneficial for the success of the alignment process, but is adverse for the success of the recognition process. We applied a factor analysis to the covariance data for the human- and model-derived face measures across the different viewpoints and found two axes that appeared consistently across all viewpoints. Projecti...

This paper reviews characteristics of human face recognition that should be reflected in any psychologically plausible computational model of face recognition. We then summarise recent results which compare aspects of human face perception and memory with the performance of two computer models which each claim some degree of biological plausibility. We show how the performance of each is correlated with human performance on the same images, but that each explains rather different aspects of human performance with these faces. We conclude with a discussion of the coding of image sequences by humans and computers. 1

. Empirical studies of face recognition suggest that faces might be stored in memory using a few canonical representations. The nature of these canonical representations is however unclear. Although psychological data show a 3/4 view advantage, physiological studies suggest profile and frontal views are stored in memory. In this paper we propose a computational approach to reconcile these findings. The patterns of results obtained when different views, or combinations of views, are used as the internal representation of a two-stage identification network consisting of an autoassociative memory followed by an rbf network are compared. Results show that 1) a frontal and a profile view are sufficient to reach the optimal network performance; 2) all the different representations produce a 3/4 view advantage, similar to that generally described for human subjects. These results indicate that although 3/4 views yield better recognition than other views, they need not be stored in memory to s...

This paper reviews processing of facial identity and expressions. The issue of independence between or two systems for these tasks has been addressed from different approaches over the past twenty-five years. More recently, neuroimaging techniques have provided researchers with new tools to investigate how facial information is processed in the brain. First, findings from "traditional" approaches to identity and expression processing are summarized. The review then