AbstractÐWe describe a new method of matching statistical models of appearance to images. A set of model parameters control modes of shape and gray-level variation learned from a training set. We construct an efficient iterative matching algorithm by learning the relationship between perturbations in the model parameters and the induced image errors. Index TermsÐAppearance models, deformable templates, model matching. 1

Abstract not found

We address the structure-from-motion problem in the context of head modeling from video sequences for which calibration data is not available. This task is made challenging by the fact that correspondences are difficult to establish due to lack of texture and that a quasi-euclidean representation is required for realism.

We present a new model-based bundle adjustment algorithm to recover the 3D model of a scene/object from a sequence of images with unknown motions. Instead of representing scene/object by a collection of isolated 3D features (usually points), our algorithm uses a surface controlled by a small set of parameters. Compared with previous modelbased approaches, our approach has the following advantages. First, instead of using the model space as a regularizer, we directly use it as our search space, thus resulting in a more elegant formulation with fewer unknowns and fewer equations. Second, our algorithm automatically associates tracked points with their correct locations on the surfaces, thereby eliminating the need for a prior 2D-to-3D association. Third, regarding face modeling, we use a very small set of face metrics (meaningful deformations) to parameterize the face geometry, resulting in a smaller search space and a better posed system. Experiments with both synthetic and real data show that this new algorithm is faster, more accurate and more stable than existing ones. Keywords: Bundle adjustment, model-based bundle adjustment, model acquisition, structure from motion, face modeling. 1.

We presentamultiview method for the computation of object shape and re ectance characteristics based on the integration of shape from shading (SFS) and stereo, for nonconstan talbedo and non-uniformly Lambertian surfaces. First we perform stereo tting on the input stereo pairs or image sequences. When the images are uncalibrated, w e recover the camera parameters using bundle adjustment. Based on the stereo result, we can automatically segment the albedo map (which istaken to be piece-wise constant) using a Minimum Description Length (MDL) based metric, to identify areas suitable for SFS (typically smooth textureless areas) and to deriv e illumination information. The shape and the illumination parameter estimates are re ned using a deformable model SFS algorithm, which iterates bet w een computing shape and illumination parameters. Our method takes into accoun tthe viewing angle dependent foreshortening and specularity e ects, and compensates as much as possible by utilizing information from more than one images. We demonstrate that we can extend the applicability of SFS algorithms to real world situations when some of its traditional assumptions are violated. We demonstrate our method by applying it to face shape reconstruction. Experimental results indicate a signi cant improvement over SFS-only or stereo-only based reconstruction. Model accuracy and detail are improved, especially in areas of low texture detail. Albedo information is retrieved and can be used to accurately re-render the model under di erent illumination conditions. 1

Generating realistic 3D human face models and facial animations has been a persistent challenge in computer graphics. We have developed a system that constructs textured 3D face models from videos with minimal user interaction. Our system takes images and video sequences of a face with an ordinary video camera. After five manual clicks on two images to tell the system where the eye corners, nose top and mouth corners are, the system automatically generates a realistic looking 3D human head model and the constructed model can be animated immediately. A user, with a PC and an ordinary camera, can use our system to generate his/her face model in a few minutes. Keywords: Face modeling, facial animation, geometric modeling, computer vision 1 Introduction One of the most interesting and difficult problems in computer graphics is the effortless generation of realistic looking, animated human face models. Animated face models are essential to computer games, film making, online chat, ...

In this paper we present a method for the integration of nonlinear holonomic constraints in deformable models and its application to the problems of shape and illuminant direction estimation from shading. Experimental results demonstrate that our method performs better than previous Shape from Shading algorithms applied to images of Lambertian objects under known illumination. It is also moregeneral as it can be applied to non-Lambertian surfaces and it does not require knowledge of the illuminant direction.

This paper presents a new method for the simultaneous estimation of lighting direction and shape from shading. The method estimates the shape and the lighting direction using a two step iterative process. We assume an initial (possibly incorrect) estimate of the lighting position. A stiff deformable model is then fitted to the image, assuming this lighting position. Next, a least-squares estimate of the lighting position is derived from the model using the Levenberg-Marquart method. The two steps --- model fitting and lighting-position estimation --- are iterated. Once the light direction has converged to a stable solution the deformable model stiffness is lowered and the model fits accurately given the lighting model. In addition, we show how the method can be used with either orthographic or perspective projection assumptions. In a variety of experiments on real and synthetic data, the method is robust to errors both to the initial light position and shape estimates. 1 Introduction ...

We present a new set of techniques for modeling and animating realistic faces from photographs and videos. Given a set of face photographs taken simultaneously, our modeling technique allows the interactive recovery of a textured 3D face model. By repeating this process for several facial expressions, we acquire a set of faces models that can be linearly combined to express a wide range of expressions. Given a video sequence, this linear face model can be used to estimate the face position, orientation, and facial expression at each frame. We illustrate these techniques on several datasets and demonstrate robust estimations of detailed face geometry and motion. 1

We show that we can effectively and automatically fit a complex facial animation model to uncalibrated image sequences. Our approach is based on model-driven bundleadjustment followed by least-squares fitting. It takes advantage of three complementary sources of information: stereo data, silhouette edges and 2--D feature points. In this way, complete head models can be acquired with a cheap and entirely passive sensor, such as an ordinary video camera. They can then be fed to existing animation software to produce synthetic sequences. 1 Introduction In earlier work [8], we have proposed an approach to fitting complex head animation models, including ears and hair, to registered stereo pairs and triplets. Here, we extend this approach so that it can take advantage of image sequences taken with a single camera, without requiring calibration data. We treat consecutive image pairs in the sequences as stereo pairs and compute disparity maps. Thus, the key issue is the estimation of the ...