Face recognition is one of the most interesting and challenging problems in computer vision. In the past, many facets of this problem have been rigorously investigated because of its importance for understanding our cognitive process and its usefulness in various applications. A great di#culty in face recognition is the separation of intrinsic facial characteristics from extrinsic image variations. Among the latter in 2D images is pose, illumination, and expression. Unfortunately, most past studies have provided variation-speci#c solutions that are not applicable to other types of variation. Performance has remained inferior to human ability and sub-optimal for practical use. This dissertation proposes a novel solution to one of these problems. We focus on processing head pose information in 2D images: analyzing, synthesizing, and identifying facial images with arbitrary pose. Successful handling of head pose variation is one of the key factors for realizing facial information processing systems in virtually any realistic and practical scenario. Our goal is twofold. One is to provide a simple and general framework whichmay be useful beyond the speci#c problem of head pose. The other is to improve the pose processing accuracy of previous studies by using this framework. xv We propose a localized two-stage linear system which is learned strictly from sample statistics and models shape and texture information separately. Instead of using variation-speci#c analytical knowledge of 3D rotation in Euclidean space, our solution utilizes a simple statistical learning framework whose applicability is not limited to the problem at hand. A wider range of head poses is covered byanumber of local linear models distributed over various poses, each of which realizes a continuous mapp...

The system presented here is a specialized version of a general object recognition system. Images of faces are represented as graphs, labeled with topographical information and local templates. Di erent poses are represented by di erent graphs. New graphs of faces are generated by an elastic graph matching procedure comparing the new face with a set of precomputed graphs: the \general face knowledge". The nal phase of the matching process can be used to generate composite images of faces and to determine certain features represented in the general face knowledge, such as gender or the presence of glasses or a beard. The graphs can be compared by a similarity function which makes the system e cient in recognizing faces. 1