We describe the Photobook system, which is a set of interactive tools for browsing and searching images and image sequences. These query tools differ from those used in standard image databases in that they make direct use of the image content rather than relying on text annotations. Direct search on image content is made possible by use of semantics-preserving image compression, which reduces images to a small set of perceptually-significant coefficients. We describe three types of Photobook descriptions in detail: one that allows search based on appearance, one that uses 2-D shape, and a third that allows search based on textural properties. These image content descriptions can be combined with each other and with textbased descriptions to provide a sophisticated browsing and search capability. In this paper we demonstrate Photobook on databases containing images of people, video keyframes, hand tools, fish, texture swatches, and 3-D medical data.

In the QBIC (Query By Image Content) project we are studying methods to query large on-line image databases using the images' content as the basis of the queries. Examples of the content we use include color, texture, and shape of image objects and regions. Potential applications include medical ("Give me other images that contain a tumor with a texture like this one"), photo-journalism ("Give me images that have blue at the top and red at the bottom"), and many others in art, fashion, cataloging, retailing, and industry. We describe a set of novel features and similarity measures allowing query by color, texture, and shape of image object. We demonstrate the effectiveness of the QBIC system with normalized precision and recall experiments on test databases containing over 1000 images and 1000 objects populated from commercially available photo clip art images, and of images of airplane silhouettes. We also consider the efficient indexing of these features, specifically addre...

We propose a method to handle approximate searching by image content in medical image databases. Image content is represented by attributed relational graphs holding features of objects and relationships between objects. The method relies on the assumption that a fixed number of "labeled" or "expected" objects (e.g., "heart", "lungs" etc.) are common in all images of a given application domain in addition to a variable number of "unexpected" or "unlabeled" objects (e.g., "tumor", "hematoma" etc.). The method can answer queries by example such as "find all X-rays that are similar to Smith's X-ray". The stored images are mapped to points in a multidimensional space and are indexed using state-of-the-art database methods (R-trees). The proposed method has several desirable properties: (a) Database search is approximate so that all images up to a prespecified degree of similarity (tolerance) are retrieved, (b) it has no "false dismissals" (i.e., all images qualifying query selection criteria are retrieved) and (c) it is much faster than sequential scanning for searching in the main memory and on the disk (i.e., by up to an order of magnitude) thus scaling-up well for large databases.

Similarity is the assessment of deviation from equivalence. Spatial similarity is complex due to the numerous constraining properties of geographic objects and their embedding in space. Among these properties, the spatial relations between geographic objects---topological, directional, and metrical---are critical, because they capture the essence of a scene's structure. These relations can be categorized as a basis for similarity assessment. This paper describes a computational method to formally assess the similarity of spatial scenes based on the ordering of spatial relations. One scene is transformed into another through a sequence of gradual changes of spatial relations. The number of changes required yields a measure that is compared against others, or against a pre-existing scale. Two scenes that require a large number of changes are less similar than scenes that require fewer changes.

The recent development of computing hardware has resulted in a rapid increase of visual information such as databases of images. To successfully utilize this increasing amount of data, we need eoeective ways to process it. Content-based image retrieval utilizes the visual content of images directly in the process of retrieving relevant images from a database. The retrieval is based on visual features such as the colors, textures, shapes, and spatial relations the image contains rather than traditional textual keywords. These features are usually extracted automatically, without the need for a human operator. In the literature survey part o...

This paper considers the requirements for the design and implementation of an image database system which supports the storage and retrieval of images by content. Attention is focused on a specific methodology for the efficient representation,indexing, and retrieval of images based on spatial relationships and properties of objects. Images are first decomposed into groups of objects and are indexed by computing addresses to all such groups. This methodology supports the efficient processing of queries by image example and avoids exhaustive searching through the entire image database. The performance of an image database system using the above methodology has been evaluated based on simulated images, as well as images obtained with computed tomography and magnetic resonance imaging. The results of this evaluation are presented and discussed.

We propose a method to handle approximate searching by image content in large image databases. Image content is represented by attributed relational graphs holding features of objects and relationships between objects. The method relies on the assumption that a fixed number of "labeled" or "expected" objects (e.g., "heart", "lungs" etc.) are common in all images of a given application domain in addition to a variable number of "unexpected" or "unlabeled" objects (e.g., "tumor", "hematoma" etc.). The method can answer queries by example such as "find all X-rays that are similar to Smith's X-ray". The stored images are mapped to points in a multidimensional space and are indexed using state-of-the-art database methods (R-trees). The proposed method has several desirable properties: (a) Database search is approximate so that all images up to a pre-specified degree of similarity (tolerance) are retrieved, (b) it has no "false dismissals" (i.e., all images qualifying query selection criteri...

A method for organizing a database of document images is discussed that is based on image data extracted from the text portion of the documents. This allows for the location of matching documents that may have been reformatted or re-imaged so that they appear significantly different. Each document image is represented by a number of descriptors that are invariant to the geometric distortions of translation, rotation, and scaling. Individual descriptors capture information about local features and the overall set of descriptors for a document image provide a redundant description of its content. Experimental results are presented that demonstrate the approach. 1.

IN THIS thesis, a new methodology is presented which supports the efficient representation, indexing and retrieval of images by content. Images may be indexed and accessed based on spatial relationships between objects, properties of individual objects, and properties of object classes. In particular, images are first decomposed into groups of objects, called "image subsets", and are indexed by computing addresses to all such groups. All groups up to a predefined maximum size are considered. This methodology supports the efficient processing of queries by image example and avoids exhaustive searching through the entire image database.

Similarity retrieval by spatial image content (i.e., using multiple objects and their relationships in space) is an open problem which has received considerable attention in the literature. The most powerful approaches of spatial image content representation and retrieval are "Attributed Relational Graphs" (ARGs) and "Symbolic Projections" (e.g., 2D Strings). In this work, a framework is proposed for studying the performance of such spatial similarity approaches in Image DataBases (IDBs). The classical ARG and 2D string matching methods are evaluated. Several variants of ARG and 2D string methods for improving their accuracy and speeding-up their time responses are also proposed and tested. A critical analysis of the performance of all these methods is presented.