We describe the Photobook system, which is a set of interactive tools for browsing and searching images and image sequences. These query tools differ from those used in standard image databases in that they make direct use of the image content rather than relying on text annotations. Direct search on image content is made possible by use of semantics-preserving image compression, which reduces images to a small set of perceptually-significant coefficients. We describe three types of Photobook descriptions in detail: one that allows search based on appearance, one that uses 2-D shape, and a third that allows search based on textural properties. These image content descriptions can be combined with each other and with textbased descriptions to provide a sophisticated browsing and search capability. In this paper we demonstrate Photobook on databases containing images of people, video keyframes, hand tools, fish, texture swatches, and 3-D medical data.

In the QBIC (Query By Image Content) project we are studying methods to query large on-line image databases using the images' content as the basis of the queries. Examples of the content we use include color, texture, and shape of image objects and regions. Potential applications include medical ("Give me other images that contain a tumor with a texture like this one"), photo-journalism ("Give me images that have blue at the top and red at the bottom"), and many others in art, fashion, cataloging, retailing, and industry. We describe a set of novel features and similarity measures allowing query by color, texture, and shape of image object. We demonstrate the effectiveness of the QBIC system with normalized precision and recall experiments on test databases containing over 1000 images and 1000 objects populated from commercially available photo clip art images, and of images of airplane silhouettes. We also consider the efficient indexing of these features, specifically addre...

We study how to measure the degree of similarity between two image contours. We propose an approach for comparing contours that takes into account deformations in object shape, the articulation of parts, and variations in the shape and size of portions of objects. Our method uses dynamic programming to compute the minimum cost of bringing one shape into the other via local deformations. Using this as a starting point, we investigate the properties that such a cost function should have to model human performance and to perform usefully in a computer vision system. We suggest novel conditions on this cost function that help capture the part-based nature of objects without requiring any explicit decomposition of shapes into their parts. We then suggest several possible cost functions based on different physical models of contours, and describe experiments with these costs. 1 Introduction Detecting similarity is a key tool in interpretating images. In this paper we develop a measure of s...

We describe a method for shape-based image database search that uses deformable prototypes to represent categories. Rather than directly comparing a candidate shape with all shape entries in the database, shapes are compared in terms of the types of nonrigid deformations (differences) that relate them to a small subset of representative prototypes. To solve the shape correspondence and alignment problem, we employ the technique of modal matching, an information-preserving shape decomposition for matching, describing, and comparing shapes despite sensor variations and nonrigid deformations. In modal matching, shape is decomposed into an ordered basis of orthogonal principal components. We demonstrate the utility of this approach for shape comparison in 2-D image databases.

: We describe Garlic, an object-oriented multimedia middleware query system. Garlic enables existing data management components, such as a relational database or a full text search engine, to be integrated into an extensible information management system that presents a common interface and user access tools. We focus in this paper on how QBIC, an image retrieval system that provides content-based image queries, can be integrated into Garlic. This results in a system in which a single query can combine visual and nonvisual data using type-specific search techniques, enabling a new breed of multimedia applications. 1 Introduction Many applications today require access to a broad range of datatypes. A patient's medical folder contains MRI scans (image), lab reports (text), doctors' dictated notes (audio), and address and insurance information (record-oriented database data). A geographic information system needs maps, satellite images, and data about roads, buildings, and populations. ...

Popular algorithms for feature matching and model extraction fall into two broad categories, generate-and-test and Hough transform variations. However, both methods suffer from problems in practical implementations. Generate-and-test methods are sensitive to noise in the data. They often fail when the generated model fit is poor due to error in the selected features. Hough transform variations are somewhat less sensitive the noise, but implementations for complex problems suffer from large time and space requirements and the detection of false positives. This paper describes a general method for solving problems where a model is extracted from or fit to data that draws benefits from both generate-and-test methods and those based on the Hough transform, yielding a method superior to both. An important component of the method is the subdivision of the problem into many subproblems. This allows efficient generateand-test techniques to be used, including the use of randomization to limit the number of subproblems that must be examined. Each subproblem is solved using pose space analysis techniques similar to the Hough transform, which lowers the sensitivity of the method to noise. This strategy is easy to implement and results in practical algorithms that are efficient and robust. We apply this method to object recognition, geometric primitive extraction, robust regression, and motion segmentation. 1

In this paper, we develop a novel multiscale stochastic image model to describe the appearance of a complex threedimensional object in a two-dimensional monochrome image. This formal image model is used in conjunction with Bayesian estimation techniques to perform automated inspection. The model is based on a stochastic tree structure in which each node is an important subassembly of the three-dimensional object. The data associated with each node or subassembly is modeled in a wavelet domain. We use a fast multiscale search technique to compute the sequential MAP (SMAP) estimate of the unknown position, scale factor, and 2-D rotation for each subassembly. The search is carried out in a manner similar to a sequential likelihood ratio test, where the process advances in scale rather than time. The results of this search determine whether or not the object passes inspection. A similar search is used in conjunction with the EM algorithm to estimate the model parameters for a given object from a set of training images. The performance of the algorithm is demonstrated on two different real assemblies.

: This paper introduces a theoretical framework for the active index. The active index facilitates the accessing and automatic manipulation of visual objects. With an active index, we can effectively and efficiently handle smart images that respond to accessing, probing, and other actions. Another important application is information retrieval in hyperspace. The active index can also be used to realize Petri nets, generalized Petri nets such as G-nets, B-trees, etc., but the dynamic nature of the active index is its most important characteristic. Implementation considerations and research issues are discussed. KEYWORDS: Active index, feature-based index, smart images, visual information systems, information retrieval in hyperspace. 1. INTRODUCTION In visual computing, an important issue is how to index visual objects, so that the visual objects can be accessed quickly and certain actions can be performed automatically. In conventional database systems, keyword-based indexing technique...

We propose a novel approach for the recognition, clustering and retrieval of shapes. The heart of our methodology is a shape matching algorithm based on dynamic programming, that operates implicitly at multiple scales, but avoids the heavy computational cost of the explicit curvature scale space representation. Our method provides for clustering, visualization and browsing of a data set, as well as for indexing, achieving up to three orders of magnitude speed-up over sequential scanning. We illustrate the application of our method to real two-dimensional static hand gesture data. We also demonstrate the superiority of our approach over traditional approaches to shape matching and retrieval, such as Fourier descriptors, Geometric and Sequential moments. Our evaluation is based on human relevance judgements following a well established methodology from the information retrieval field. 1 Introduction Object recognition is an important problem in computer vision and has received ...

We describe the Photobook system, which is a set of interactive tools for browsing and searching images and image sequences. These query tools differ from those used in standard image databases in that they make direct use of the image content rather than relying on text annotations. Direct search on image content is made possible by use of semantics-preserving image compression, which reduces images to a small set of perceptually-significant coefficients. We describe three types of Photobook descriptions in detail: one that allows search based on appearance, one that uses 2-D shape, and a third that allows search based on textural properties. These image content descriptions can be combined with each other and with textbased descriptions to provide a sophisticated browsing and search capability. In this paper we demonstrate Photobook on databases containing images of people, video keyframes, hand tools, fish, texture swatches, and 3-D medical data. 1 Introduction: The Problem Digita...