We present new techniques for creating photorealistic textured 3D facial models from photographs of a human subject, and for creating smooth transitions between different facial expressions by morphing between these different models. Starting from several uncalibrated views of a human subject, we employ a user-assisted technique to recover the camera poses corresponding to the views as well as the 3D coordinates of a sparse set of chosen locations on the subject's face. A scattered data interpolation technique is then used to deform a generic face mesh to fit the particular geometry of the subject's face. Having recovered the camera poses and the facial geometry, we extract from the input images one or more texture maps for the model. This process is repeated for several facial expressions of a particular subject. To generate transitions between these facial expressions we use 3D shape morphing between the corresponding face models, while at the same time blending the corresponding tex...

We describe an image based rendering approach that generalizes many image based rendering algorithms currently in use including light field rendering and view-dependent texture mapping. In particular it allows for lumigraph style rendering from a set of input cameras that are not restricted to a plane or to any specific manifold. In the case of regular and planar input camera positions, our algorithm reduces to a typical lumigraph approach. In the case of fewer cameras and good approximate geometry, our algorithm behaves like view-dependent texture mapping. Our algorithm achieves this flexibility because it is designed to meet a set of desirable goals that we describe. We demonstrate this flexibility with a variety of examples. Keyword Image-Based Rendering 1

A surface light field is a function that assigns a color to each ray originating on a surface. Surface light fields are well suited to constructing virtual images of shiny objects under complex lighting conditions. This paper presents a framework for construction, compression, interactive rendering, and rudimentary editing of surface light fields of real objects. Generalizations of vector quantization and principal component analysis are used to construct a compressed representation of an object's surface light field from photographs and range scans. A new rendering algorithm achieves interactive rendering of images from the compressed representation, incorporating view-dependent geometric level-of-detail control. The surface light field representation can also be directly edited to yield plausible surface light fields for small changes in surface geometry and reflectance properties.

In image-based rendering, images acquired from a scene are used to represent the scene itself. A number of reference images are required to fully represent even the simplest scene. This leads to a number of problems during image acquisition and subsequent reconstruction. We present the multiple-center-of-projection image, a single image acquired from multiple locations, which solves many of the problems of working with multiple range images. This work develops and discusses multiple-center-ofprojection images, and explains their advantages over conventional range images for image-based rendering. The contributions include greater flexibility during image acquisition and improved image reconstruction due to greater connectivity information. We discuss the acquisition and rendering of multiple-center-of-projection datasets, and the associated sampling issues. We also discuss the unique epipolar and correspondence properties of this class of image. CR Categories: I.3.3 [Computer Graphic...

Abstract not found

Creating realistic images has been a major focus in the study of computer graphics for much of its history. This e ort has led to mathematical models and algorithms that can compute predictive, or physically realistic, images from known camera positions and scene descriptions that include the geometry of objects, the re ectance of surfaces, and the lighting used to illuminate the scene. These images accurately describe the physical quantities that would be measured from a real scene. Because these algorithms can predict real images, they can also be used in inverse problems to work backward from photographs to attributes of the scene. Work on three such inverse rendering problems is described. The rst, inverse lighting, assumes knowledge of geometry, re ectance, and the recorded photograph and solves for the lighting in the scene. A technique using a linear least-squares system is proposed and demonstrated. Also demonstrated is an application of inverse lighting, called re-lighting, which modi es lighting in photographs. The second two inverse rendering problems solve for unknown re ectance, given images with known geometry, lighting, and camera positions. Photographic texture measurement

Image-based and model-based methods are two representative rendering methods for generating virtual images of objects from their real images. Extensive research on these two methods has been made in CV and CG communities. However, both methods still have several drawbacks when it comes to applying them to the mixed reality where we integrate such virtual images with real background images. To overcome these difficulties, we propose a new method, which we refer to as the Eigen-Texture method. The proposed method samples appearances of a real object under various illumination and viewing conditions, and compresses them in the 2D coordinate system defined on the 3D model surface. The 3D model is generated from a sequence of range images. The Eigen-Texture method is practical because it does not require any detailed reflectance analysis of the object surface, and has great advantages due to the accurate 3D geometric models. This paper describes the method, and reports on its implementation...

. In this paper we propose a new approach for mapping and blending textures on 3D geometries. The system starts from a 3D mesh which represents a real object and improves this model with pictorial detail. Texture detail is acquired via a common photographic process directly from the real object. These images are then registered and stitched on the 3D mesh, by integrating them into a single standard texture map. An optimal correspondence between regions of the 3D mesh and sections of the acquired images is built. Then, a new approach is proposed to produce a smooth join between different images that map on adjacent sections of the surface, based on texture blending. For each mesh face which is on the adjacency border between different observed images, a corresponding triangular texture patch is resampled as a weighted blend of the corresponding adjacent images sections. The accuracy of the resampling and blending process is improved by computing an accurate piecewise local r...

In this paper a system is presented which automatically registers and stitches textures acquired from multiple photographic images onto the surface of a given corresponding 3D model. Within this process the camera position, direction and field of view must be determined for each of the images. For this registration, which aligns a 2D image to a 3D model we present an efficient hardware-accelerated silhouette-based algorithm working on different image resolutions that accurately registers each image without any user interaction. Besides the silhouettes, also the given texture information can be used to improve accuracy by comparing one stitched texture to already registered images resulting in a global multi-view optimization. After the 3D-2D registration for each part of the 3D model's surface the view is determined which provides the best available texture. Textures are blended at the borders of regions assigned to different views.

This paper describes the successful combination of pre-generated and dynamically updated image-based representations to accelerate the visualization of complex virtual environments. We introduce a new type of impostor, which has the desirable property of limiting de-occlusion errors to a user-specified amount. This impostor, composed of multiple layers of textured meshes, replaces the distant geometry and is much faster to draw. It captures the relevant depth complexity in the model without resorting to a complete sampling of the scene. We show that layers can be dynamically updated during visualization. This guarantees bounded scene complexity in each frame and also exploits temporal coherence to improve image quality when possible. We demonstrate the strengths of this approach in the context of city walkthroughs.