A major unsolved problem in computer graphics is the construction and animation of realistic human facial models. Traditionally, facial models have been built painstakingly by manual digitization and animated by ad hoc parametrically controlled facial mesh deformations or kinematic approximation of muscle actions. Fortunately, animators are now able to digitize facial geometries through the use of scanning range sensors and animate them through the dynamic simulation of facial tissues and muscles. However, these techniques require considerableuser input to construct facial models of individuals suitable for animation. In this paper, we present a methodology for automating this challenging task. Starting with a structured facial mesh, we develop algorithms that automatically construct functional models of the heads of human subjects from laser-scanned range and reflectance data. These algorithms automatically insert contractile muscles at anatomically correct positions within a dynamic skin model and root them in an estimated skull structure with a hinged jaw. They also synthesize functional eyes, eyelids, teeth, and a neck and fit them to the final model. The constructed face may be animated via muscle actuations. In this way, we create the most authentic and functional facial models of individuals available to date and demonstrate their use in facial animation.

We describe a computer vision system for observing facial motion by using an optimal estimation optical flow method coupled with geometric, physical and motion-based dynamic models describing the facial structure. Our method produces a reliable parametric representation of the face's independent muscle action groups, as well as an accurate estimate of facial motion. Previous efforts at analysis of facial expression have been based on the Facial Action Coding System (FACS), a representation developed in order to allow human psychologists to code expression from static pictures. To avoid use of this heuristic coding scheme, we have used our computer vision system to probabilistically characterize facial motion and muscle activation in an experimental population, thus deriving a new, more accurate representation of human facial expressions that we call FACS+. Finally, we show how this method can be used for coding, analysis, interpretation, and recognition of facial expressions.

This paper presents a survey of the work done in modeling deformable objects within the computer graphics research community. The research has a long history and a wide variety of approaches have been used. This paper organizes the diversity of research by the technique used rather than by the application, although applications are discussed throughout. This paper presents some purely geometric approaches for modeling deformable objects, but focuses on physically based approaches. In the latter category are mass-spring models, nite element models, approximate continuum models, and low degree of freedom models. Special emphasis is placed on nite element models, which o er the greatest accuracy, but have seen limited use in computer graphics. The paper also suggests important areas for future research. 1

We describe a computer vision system for observing the "action units" of a face using video sequences as input. The visual observation (sensing) is achieved by using an optimal estimation optical flow method coupled with a geometric and a physical (muscle) model describing the facial structure. This modeling results in a time-varying spatial patterning of facial shape and a parametric representation of the independent muscle action groups, responsible for the observed facial motions. These muscle action patterns may then be used for analysis, interpretation, and synthesis. Thus, by interpreting facial motions within a physics-based optimal estimation framework, a new control model of facial movement is developed. The newly extracted action units (which we name "FACS+") are both physics and geometry-based, and extend the well-known FACS parameters for facial expressions by adding temporal information and non-local spatial patterning of facial motion.

The human face is an elastic object. A natural paradigm for representing facial expressions is to form a complete 3D model of facial muscles and tissues. However, determining the actual parameter values for synthesizing and animating facial expressions is tedious; evaluating these parameters for facial expression analysis out of grey-level images is ahead of the state of the art in computer vision. Using only 2D face images and a small number of anchor points, we show that the method of radial basis functions provides a powerful mechanism for processing facial expressions. Although constructed specifically for facial expressions, our method is applicable to other elastic objects as well.

Hand and face gestures are modeled using an appearance-based approach in which patterns are represented as a vector of similarity scores to a set of view models defined in space and time. These view models are learned from examples using unsupervised clustering techniques. A supervised learning paradigm is used to interpolate view scores into a task-dependent coordinate system appropriate for recognition and control tasks. We apply this analysis to the problem of context-specific gesture interpolation and recognition, and demonstrate real-time systems which perform these tasks.

This thesis describes a computer vision system for observing the "action units" of a face using video sequences as input. The visual observation (sensing) is achieved by using an optimal estimation optical flow method coupled with a geometric and a physical (muscle) model describing the facial structure. This modeling results in a time-varying spatial patterning of facial shape and a parametric representation of the independent muscle action groups responsible for the observed facial motions. These muscle action patterns are then used for analysis, interpretation, recognition, and synthesis of facial expressions. Thus, by interpreting facial motions within a physics-based optimal estimation framework, a new control model of facial movement is developed. The newly extracted action units (which we name "FACS+") are both physics and geometry-based, and extend the well known FACS parameters for facial expressions by adding temporal information and non-local spatial patterning of facial mot...

This paper develops a highly automated approach to constructing realistic, working models of human heads for use in animation. These physics-based models are anatomically accurate and may be made to conform closely to specific individuals. We begin by scanning a person with a laser sensor which circles around the head, acquiring detailed range and reflectance information. Next, an automatic conformation algorithm adapts a triangulated face mesh of predetermined topological structure to these data. The generic mesh, which is reusable with different individuals, reduces the range data to an efficient, polygonal approximation of the facial geometry and supports a high-resolution texture mapping of the skin reflectivity. The conformed polygonal mesh forms the epidermal layer of a new, physics-based model of facial tissue. An automatic algorithm constructs the multilayer synthetic skin and estimates an underlying rigid "skull" substructure with a jointed jaw. Finally, the algorithm inserts synthetic muscles into the deepest layer of the facial tissue. These contractile actuators, which emulate the primary muscles of facial expression, generate forces that deform the synthetic tissue into meaningful expressions. To increase realism, we include constraints to emulate tissue incompressibility and to enable the tissue to slide over the skull substructure without penetrating into it. The resulting animate models appear significantly more realistic than our previous physics-based facial models. Keywords: Physics-Based Facial Modeling, Facial Animation, Cylindrical Facial Scanning, Feature-Based Facial Adaptation, Texture Mapping, Discrete Deformable Models. 1

The correspondence problem in computer vision is basically a matching task between two or more sets of features. Computing feature correspondence is of great importance in computer vision, especially in the subfields of object recognition, stereo, and motion. In this paper, weintroduce a vectorized image representation, which is a feature-based representation where correspondence has been established with respect to a reference image. The representation consists of two image measurements made at the feature points: shape and texture. Feature geometry, or shape, is represented using the (x# y) locations of features relative to the some standard reference shape. Image grey levels, or texture, are represented by mapping image grey levels onto the standard reference shape. Computing this representation is essentially a correspondence task, and in this paper we explore an automatic technique for "vectorizing" face images.

n the next generation of animation systems, motion control tends to be performed automatically using A.I. and robotics techniques. In particular, motion is planned at a task level and computed using physical laws. More recently, researchers have developed models of behavioral animation and simulation of autonomous creatures. State variables and evolution laws Computer animation may be defined as a technique in which the illusion of movement is created by displaying on a screen, or recording on a recording device a series of individual states of a dynamic scene. Formally, any computer animation sequence may be defined as a set of objects characterized by state variables evolving over time. For example, a human character is normally characterized using its joint angles as state variables. To improve computer animation, attention needs to be devoted to the design of evolution laws [Magnenat Thalmann and Thalmann, 1985]. Animators must be able to apply any evolution law to the state vari