. We present here an implementation of NeTra, a prototype image retrieval system that uses color, texture, shape and spatial location information in segmented image regions to search and retrieve similar regions from the database. A distinguishing aspect of this system is its incorporation of a robust automated image segmentation algorithm that allows object- or region-based search. Image segmentation significantly improves the quality of image retrieval when images contain multiple complex objects. Images are segmented into homogeneous regions at the time of ingest into the database, and image attributes that represent each of these regions are computed. In addition to image segmentation, other important components of the system include an efficient color representation, and indexing of color, texture, and shape features for fast search and retrieval. This representation allows the user to compose interesting queries such as "retrieve all images that contain regions that have the colo...

. We explore the effect of dimensionality on the "nearest neighbor " problem. We show that under a broad set of conditions (much broader than independent and identically distributed dimensions), as dimensionality increases, the distance to the nearest data point approaches the distance to the farthest data point. To provide a practical perspective, we present empirical results on both real and synthetic data sets that demonstrate that this effect can occur for as few as 10-15 dimensions. These results should not be interpreted to mean that high-dimensional indexing is never meaningful; we illustrate this point by identifying some high-dimensional workloads for which this effect does not occur. However, our results do emphasize that the methodology used almost universally in the database literature to evaluate high-dimensional indexing techniques is flawed, and should be modified. In particular, most such techniques proposed in the literature are not evaluated versus simple...

A new scheme of learning similarity measure is proposed for content-based image retrieval (CBIR). It learns a boundary that separates the images in the database into two clusters. Images inside the boundary are ranked by their Euclidean distances to the query. The scheme is called constrained similarity measure (CSM), which not only takes into consideration the perceptual similarity between images, but also significantly improves the retrieval performance of the Euclidean distance measure. Two techniques, support vector machine (SVM) and AdaBoost from machine learning, are utilized to learn the boundary. They are compared to see their differences in boundary learning. The positive and negative examples used to learn the boundary are provided by the user with relevance feedback. The CSM metric is evaluated in a large database of 10 009 natural images with an accurate ground truth. Experimental results demonstrate the usefulness and effectiveness of the proposed similarity measure for image retrieval.

In this paper we describe a probabilistic image matching scheme in which the image representation is continuous and the similarity measure and distance computation are also defined in the continuous domain. Each image is first represented as a Gaussian mixture distribution and images are compared and matched via a probabilistic measure of similarity between distributions. A common probabilistic and continuous framework is applied to the representation as well as the matching process, ensuring an overall system that is theoretically appealing. Matching results are investigated and the application to an image retrieval system is demonstrated.

Accessing large image databases requires effective indexing in order to restrict the number of database items that have to be processed. Indexing based on shapes is particularly challenging owing to the difficulty of deriving a similarity measure that supports clustering of shapes conforming with human perceptual similarity. Most previous techniques are based on the extraction of salient shape features and their organization into multi-dimensional point access structures. However, these features are extracted by analyzing shapes at a single resolution scale, and are not able to provide a robust representation. In this paper, we present a technique which exploits multi-scale analysis of shapes, to derive a hierarchical shape representation in which shape details are progressively filtered out while shape characterizing elements are preserved. A graph structure is introduced to represent shape parts at different scales and a procedure is defined to merge graphs of different shapes. Given a query shape, the graph can be traversed to select, through a coarse to fine matching, those database shapes which share similar structural parts with the query. # 1999 Elsevier Science B.V. All rights reserved.

A distributed multimedia information system allows applications to access a variety of data, of different modalities, stored in data sources with their own specialized search capabilities. In such a system, the user can request that a set of objects be ranked by a particular property, or by a combination of properties. In [Fag96], Fagin gives an algorithm for efficiently merging multiple ordered streams of ranked results, to form a new stream ordered by a combination of those ranks. In this paper, we describe the implementation of Fagin's algorithm in the Garlic middleware system [C + 95], including a novel, incremental version of the algorithm well-suited to an environment in which users are dynamically exploring their data. While the algorithm is a simple one, designed with the distributed environment in mind, we found that the assumptions it makes about random access limit its applicability dramatically. Having implemented Fagin's algorithm, we compared it with other exec...

Abstract—Content-based retrieval of multimedia database calls for content-based indexing techniques. Different from conventional databases, where data items are represented by a set of attributes of elementary data types, multimedia objects in multimedia databases are represented by a collection of features; similarity of object contents depends on context and frame of reference; and features of objects are characterized by multimodal feature measures. These lead to great challenges for content-based indexing. On the other hand, there are special requirements on content-based indexing: To support visual browsing, similarity retrieval, and fuzzy retrieval, nodes of the index should represent certain meaningful categories. That is to say that certain semantics must be added when performing indexing. ContIndex, the context-based indexing technique presented in this paper, is proposed to meet these challenges and special requirements. The indexing tree is formally defined by adapting a classification-tree concept. Horizontal links among nodes in the same level enhance the flexibility of the index. A special neural-network model, called Learning based on Experiences and Perspectives (LEP), has been developed to create node categories by fusing multimodal feature measures. It brings into the index the capability of self-organizing nodes with respect to certain context and frames of reference. An icon image is generated for each intermediate node to facilitate visual browsing. Algorithms have been developed to support multimedia object archival and retrieval using ContIndex. ContIndex has been successfully applied to two applications: A facial image retrieval system, CAFIIR, and a trademark archival and registration system, STAR. Index Terms—Indexing, content-based retrieval, multimedia, image database, image analysis, neural networks, fusion of multiple feature measures. 1

In this paper we combine image feature extraction with indexing techniques for efficient retrieval in large texture images databases. A 2D image signal is processed using a set of Gabor filters to derive a 120 component feature vector representing the image. The feature components are ordered based on the relative importance in characterizing a given texture pattern, and this facilitates the development of efficient indexing mechanisms. We propose three different sets of indexing features based on the "best feature", the average feature and a combination of both. We investigate the tradeoff between accuracy and discriminating power using these different indexing approaches, and conclude that the combination of "best feature" and the average feature gives the best results. Keywords: Texture Images, Gabor Filters, Multi-dimensional Indexing, Indexing Structures. 1 INTRODUCTION Searching a large database of image data poses many challenging problems. In particular, issues related to ima...

This paper describes a content-based image retrieval digital library that supports geographical image retrieval over a testbed of 800 aerial photographs, each 25 megabytes in size. In addition, this paper also introduces a methodology to evaluate the performance of the algorithms in the prototype system. The major contributions of this paper are two. 1) We suggest an approach that incorporates various image processing techniques including Gabor filters, image enhancement, and image compression, as well as information analysis technique such as self-organizing map (SOM) into an effective large-scale geographical image retrieval system. 2) We present two experiments that evaluate the performance of the Gaborfilter -extracted features along with the corresponding similarity measure against that of human perception, addressing the lack of studies in assessing the consistency between an image representation algorithm or an image categorization method and human mental model.

This thesis presents an investigation of methods for conducting an efficient look-up in a pictorial "phonebook" (i.e., a facial image database). Although research on efficient "mug-shot search" is under way, little has yet been done to evaluate the effectiveness of various proposed techniques, and much work remains before systems as practical or ubiquitous as phonebooks are attainable. The thesis describes a prototype system based on the idea of combining a composite face creation method with a face-recognition technique, so that a user may create a facial image and then automatically locate other similar-looking faces in the database. Several methods for evaluating such a system are presented as well as the results and analysis of a user-study employing the methods. Three basic system components are considered and evaluated: the metric for determining which faces are most similar in appearance to a given "query" face, the interface for producing the query face, and the search strateg...