We describe the Photobook system, which is a set of interactive tools for browsing and searching images and image sequences. These query tools differ from those used in standard image databases in that they make direct use of the image content rather than relying on text annotations. Direct search on image content is made possible by use of semantics-preserving image compression, which reduces images to a small set of perceptually-significant coefficients. We describe three types of Photobook descriptions in detail: one that allows search based on appearance, one that uses 2-D shape, and a third that allows search based on textural properties. These image content descriptions can be combined with each other and with textbased descriptions to provide a sophisticated browsing and search capability. In this paper we demonstrate Photobook on databases containing images of people, video keyframes, hand tools, fish, texture swatches, and 3-D medical data.

A new region-based approach to nonrigid motion tracking is described. Shape is defined in terms of a deformable triangular mesh that captures object shape plus a color texture map that captures object appearance. Photometric variations are also modeled. Nonrigid shape registration and motion tracking are achieved by posing the problem as an energy-based, robust minimization procedure. The approach provides robustness to occlusions, wrinkles, shadows, and specular highlights. The formulation is tailored to take advantage of texture mapping hardware available in many workstations, PC's, and game consoles. This enables nonrigid tracking at speeds approaching video rate. 1 Introduction A key open problem in tracking is that of encoding and comparing shapes as they undergo nonrigid deformation. Simply providing robustness to nonrigid deformation is insufficient, because deformation often provides important information about how shapes are related. To make things worse, tracking must also ...

Abstract not found

This paper describes a method of combining two approaches to modelling flexible objects. Modal Analysis using Finite Element Methods (FEMs) generates a set of vibrational modes for a single shape. Point Distribution Models (PDMs) generate a statistical model of shape and shape variation from a set of example shapes. A new approach is described which generates vibrational modes when few example shapes are available and changes smoothly to using more statistical modes of variation when a large data set is presented. Results are given for both synthetic and real examples. Experiments using the models for image search show that the combined version performs better than either the PDM or FEM models alone. 1

We describe a novel approach for image matching based on deformable intensity surfaces. In this approach, the intensity surface of the image is modeled as a deformable 3D mesh in the (x; y; I(x;y)) space. Each surface point has 3 degrees of freedom, thus capturing fine surface changes. A set of representative deformations within a class of objects (e.g. faces) are statistically learned through a Principal Components Analysis, thus providing a priori knowledge about object-specific deformations. We demonstrate the power of the approach by examples such as image matching and interpolation of missing data. Moreover this approach dramatically reduces the computational cost of solving the governing equation for the physically based system by approximately three orders of magnitude.

. The goal of this work is to develop an approach to shape representation and classification that will allow us to detect and quantify di#erences in shape of anatomical structures due to various disorders. We used a robust version of skeletons for feature extraction and linear discriminant analysis ( the Fisher linear discriminant and the linear Support Vectors method) for classification. We propose a way to map the classification results back into the image domain, interpreting shape differences as a deformation required to bring a shape from one class to the other. An example of analyzing corpus callosum shape in schizophrenia is reported, as well as the results of the study of the statistical properties of the classifier using cross validation techniques. 1 Introduction Our goal is to build a framework for statistical shape analysis using classification techniques applied to feature descriptors. We perform shape feature extraction using skeletons. To make the process of ...

AbstractÐA method for deformable shape detection and recognition is described. Deformable shape templates are used to partition the image into a globally consistent interpretation, determined in part by the minimum description length principle. Statistical shape models enforce the prior probabilities on global, parametric deformations for each object class. Once trained, the system autonomously segments deformed shapes from the background, while not merging them with adjacent objects or shadows. The formulation can be used to group image regions obtained via any region segmentation algorithm, e.g., texture, color, or motion. The recovered shape models can be used directly in object recognition. Experiments with color imagery are reported. Index TermsÐImage segmentation, region merging, object detection and recognition, deformable templates, nonrigid shape models, statistical shape models. 1

this paper, we develop a hierarchical statistical modeling framework for the representation, segmentation, and tracking of 2D deformable structures in image sequences. The model relies on the specification of a template, on which global as well as local deformations are defined. Global deformations are modeled using a statistical modal analysis of the deformations observed on a representative population. Local deformations are represented by a (first-order) Markov random process. A model-based segmentation of the scene is obtained by a joint bayesian estimation of global deformation parameters and local deformation variables. Spatial or spatio-temporal observations are considered in this estimation procedure, yielding an edge-based or a motion-based segmentation of the scene. The segmentation procedure is combined with a temporal tracking of the deformable structure over long image sequences, using a Kalman filtering approach. This combined segmentationtracking procedure has produced reliable extraction of deformable parts from long image sequences in adverse situations such as low signal-to-noise ratio, nongaussian noise, partial occlusions, or random initialization. The approach is demonstrated on a variety of synthetic as well as real-world image sequences featuring different classes of deformable objects

We address the problem of tracking and reconstructing 3D human lip motions from a 2D view. This problem is challenging due both to the complex nature of lip motions and the minimal data available from a raw video stream of the face. We counter both of these difficulties with statistical approaches. We first build a physically-based 3D model of lips and train it to cover only the subspace of lip motions. We then track this model in video by finding the shape within the subspace that maximizes the posterior probability of the model given the observed features. In this study, the features are the likelihoods of the lip and non-lip color classes: we iteratively derive forces from these values to apply to the physical model and converge to the final solution. Because of the full 3D nature of the model, this framework allows us to track the lips from any head pose. In addition, because of the constraints imposed by the learned subspace of the model, we are able to accurately estimate the full 3D lip shape from the 2D view.

We have previously described modal analysis, an efficient, physically-based solution for recovering, tracking, and recognizing solid models from 2-D and 3-D sensor data. The underlying representation consists of two levels: modal deformations, which describe the overall shape of a solid, and displacement maps, which employ a multiscale wavelet representation to provide local and fine surface detail. This paper addresses the problem of recovering modal models in the underconstrained case of fitting a 3-D model to contours found in medical slice and X-ray data. We will describe an extension which can be used to incorporate measurement uncertainty while estimating the modal deformation parameters. Finally, we give details about how to compress dense 3-D point data from surfaces, by use of displacement maps and wavelets.