The paper presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems. Step one of the review is image processing for retrieval sorted by color, texture, and local geometry. Features for retrieval are discussed next, sorted by: accumulative and global features, salient points, object and shape features, signs, and structural combinations thereof. Similarity of pictures and objects in pictures is reviewed for each of the feature types, in close connection to the types and means of feedback the user of the systems is capable of giving by interaction. We briefly discuss aspects of system engineering: databases, system architecture, and evaluation. In the concluding section, we present our view on: the driving force of the field, the heritage from computer vision, the influence on computer vision, the role of similarity and of interaction, the need for databases, the problem of evaluation, and the role of the semantic gap.

Abstract not found

Color is a feature of the great majority of content-based image retrieval systems. However the robustness, effectiveness, and efficiency of its use in image indexing are still open issues. This paper provides a comprehensive survey of the methods for color image indexing and retrieval described in the literature. In particular, image preprocessing, the features used to represent color information, and the measures adopted to compute the similarity between the features of two images are critically analyzed.

We propose a new method for automated large scale gathering of Web images relevant to specified concepts. Our main goal is to build a knowledge base associated with as many concepts as possible for large scale object recognition studies. A second goal is supporting the building of more accurate text-based indexes for Web images. In our method, good quality candidate sets of images for each keyword are gathered as a function of analysis of the surrounding HTML text. The gathered images are then segmented into regions, and a model for the probability distribution of regions for the concept is computed using an iterative algorithm based on the previous work on statistical image annotation. The learned model is then applied to identify which images are visually relevant to the concept implied by the keyword. Implicitly, which regions or the images are relevant is also determined. Our experiments reveal that the new method performs much better than Google Image Search and a simple method based on more standard content based image retrieval methods.

With the proliferation of image data, the need to search and retrieve images efficiently and accurately from a large image database or a collection of image databases has drastically increased. To address such a demand, a unified framework called Markov Model Mediators (MMMs) is proposed in this paper to facilitate conceptual database clustering and to improve the query processing performance by analyzing the summarized knowledge. The unique characteristics of MMMs are that it provides the capabilities of exploring the affinity relations among the images at the database level and among the databases at the cluster level respectively, using an effective data mining process. At the database level, each database is modeled by an intra-database MMM which enables accurate image retrieval within the database. Then the conceptual database clustering is performed and cluster-level knowledge summarization is conducted to reduce the cost of retrieving images across the databases. This framework has been tested using a set of image databases, which contain various numbers of images with different dimensions and concept categories. The experimental results demonstrate that our framework achieves better retrieval accuracy via inter-cluster retrieval than that of intra-cluster retrieval with minimal extra effort.

We exploit the retrieval of visual information from biomedical scientific publication databases. Therefore, we consider the use of domain specific genres to automatically subdivide large image databases into smaller, consistent parts. Combination with Latent Semantic Indexing on the picture captions allows for efficient retrieval of images in specific categories. We demonstrate our approach on a large collection of images with captions from the Elsevier Brain Research publications. Initial result demonstrate the power of the proposed combination.

In this paper, we propose building a vertical image search engine called EnjoyPhoto that leverages rich metadata from various photo forum web sites to meet users ’ requirements for enjoying high-quality photos, which is virtually impossible in traditional image search engines. To solve the ranking problem when aggregating multiple photo forums, we propose a novel rank fusion algorithm that uses duplicate photos to normalize rating scores. To further improve user experiences in enjoying photos, we design an in-place image browsing interface, and compare it with several other interfaces in a user study. With rich metadata and rating information, more attractive user interfaces are enabled, including slideshow authoring and photo recommendations. We conducted experiments and user studies on a 2.5-million image database to evaluate the proposed rank fusion algorithm, investigate the rationale behind building a vertical image search engine, and study user interfaces and preferences for the purpose of enjoying high-quality photos. The experimental results demonstrate the effectiveness of the proposed ranking algorithm. The results also show that the 2.5-million high-quality image database in EnjoyPhoto performs comparably with Google’s 1billion image database for queries related to location, nature, and daily life categories. Finally, our results show that the in-place browsing interface—called Force-Transfer view—is much more convenient for users than traditional interfaces.

Because of recent advances in graphics hard- and software, both the production and use of 3D models are increasing at a rapid pace. As a result, a large number of 3D models have become available on the web, and new research is being done on 3D model retrieval methods. Query and retrieval can be done solely based on associated text, as in image retrieval, for example (e.g. Google Image Search [1] and [2, 3]). Other research focuses on shape-based retrieval, based on methods that measure shape similarity between 3D models (e.g., [4]).

Integrated' classification refers to the conjunctive or competitive use of two or more (neural) classifiers. A cooperative neural network system comprising two independently trained Kohonen networks and co-operating with the help of a Hebbian network, is described. The effectiveness of such a network is demonstrated by using it to retrieve images and related texts from a multi-media database. Preliminary results of such an approach appear to be encouraging.

Content based indexing of multimedia has always been a challenging task. The enormity and the diversity of the multimedia content on the web adds another dimension to this challenge. In this paper, we examine ways of combining visual and textual information for content based indexing of multimedia on the web. In particular, we examine different methods of combining evidences due to face detection, Text/HTML analysis and face recognition for identifying person images. We provide experimental evaluation of the following strategies: i) Face detection on the image followed by Text/HTML analysis of the containing page; ii) face detection followed by face recognition; iii) face detection followed by a linear combination of evidences due to text/HTML analysis and face recognition; and iv) face detection followed by a Dempster-Shafer combination of evidences due to text/HTML analysis and face recognition. These strategies were implemented in an automatic web search agent named Diogenes 1 and...