The paper presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems. Step one of the review is image processing for retrieval sorted by color, texture, and local geometry. Features for retrieval are discussed next, sorted by: accumulative and global features, salient points, object and shape features, signs, and structural combinations thereof. Similarity of pictures and objects in pictures is reviewed for each of the feature types, in close connection to the types and means of feedback the user of the systems is capable of giving by interaction. We briefly discuss aspects of system engineering: databases, system architecture, and evaluation. In the concluding section, we present our view on: the driving force of the field, the heritage from computer vision, the influence on computer vision, the role of similarity and of interaction, the need for databases, the problem of evaluation, and the role of the semantic gap.

This paper provides a comprehensive survey of the technical achievements in the research area of image retrieval, especially content-based image retrieval, an area that has been so active and prosperous in the past few years. The survey includes 100+ papers covering the research aspects of image feature representation and extraction, multidimensional indexing, and system design, three of the fundamental bases of content-based image retrieval. Furthermore, based on the state-of-the-art technology available now and the demand from real-world applications, open research issues are identified and future promising research directions are suggested. C ○ 1999 Academic Press 1.

We have witnessed great interest and a wealth of promise in content-based image retrieval as an emerging technology. While the last decade laid foundation to such promise, it also paved the way for a large number of new techniques and systems, got many new people involved, and triggered stronger association of weakly related fields. In this article, we survey almost 300 key theoretical and empirical contributions in the current decade related to image retrieval and automatic image annotation, and in the process discuss the spawning of related subfields. We also discuss significant challenges involved in the adaptation of existing image retrieval techniques to build systems that can be useful in the real world. In retrospect of what has been achieved so far, we also conjecture what the future may hold for image retrieval research.

Abstract—Grouping images into (semantically) meaningful categories using low-level visual features is a challenging and important problem in content-based image retrieval. Using binary Bayesian classifiers, we attempt to capture high-level concepts from low-level image features under the constraint that the test image does belong to one of the classes. Specifically, we consider the hierarchical classification of vacation images; at the highest level, images are classified as indoor or outdoor; outdoor images are further classified as city or landscape; finally, a subset of landscape images is classified into sunset, forest, and mountain classes. We demonstrate that a small vector quantizer (whose optimal size is selected using a modified MDL criterion) can be used to model the class-conditional densities of the features, required by the Bayesian methodology. The classifiers have been designed and evaluated on a database of 6931 vacation photographs. Our system achieved a classification accuracy of 90.5 % for indoor/outdoor, 95.3 % for city/landscape, 96.6 % for sunset/forest & mountain, and 96 % for forest/mountain classification problems. We further develop a learning method to incrementally train the classifiers as additional data become available. We also show preliminary results for feature reduction using clustering techniques. Our goal is to combine multiple two-class classifiers into a single hierarchical classifier. Index Terms—Bayesian methods, content-based retrieval, digital libraries, image content analysis, minimum description length, semantic

This paper provides a comprehensive survey of the technical achievements in the research area of Image Retrieval, especially Content-Based Image Retrieval, an area so active and prosperous in the past few years. The survey includes 100+ papers covering the research aspects of image feature representation and extraction, multi-dimensional indexing, and system design, three of the fundamental bases of Content-Based Image Retrieval. Furthermore, based on the state-of-the-art technology available now and the demand from real-world applications, open research issues are identified, and future promising research directions are suggested. 1. INTRODUCTION Recent years have seen a rapid increase of the size of digital image collections. Everyday, both military and civilian equipment generates giga-bytes of images. Huge amount of information is out there. However, we can not access to or make use of the information unless it is organized so as to allow efficient browsing, searching and retriev...

This paper describes the architecture and the salient characteristics of the image database system El Nino. The system uses a new interaction model which purports to overcome the problem of the semantic gap. We have a semantic gap when the meaning that the user has in mind for an image is at a higher semantic level than the features on which the database operates. We argue that we can overcome the problems caused by the semantic gap if we replace the usual query paradigm with a more active exploration process, and develop an interface based on these premises. Other relevant aspects of El Nino, which are discussed in the paper are its distributed architecture composed of partially independent engines connected to a mediator, its attempts to integrate visual and textual queries, and its query algebra approach to the problem of putting together queries from different engines. 1 Introduction What is exactly wrong with existing image databases? Most people would wholeheartedly agree that s...

In many vision applications, the practice of supervised learning faces several difficulties, one of which is that insufficient labeled training data result in poor generalization. In image retrieval, we have very few labeled images from query and relevance feedback so that it is hard to automatically weight image features and select similarity metrics for image classification. This paper investigates the possibility of including an unlabeled data set to make up the insufficiency of labeled data. Different from most current research in image retrieval, the proposed approach tries to cast image retrieval as a transductive learning problem, in which the generalization of an image classifier is only defined on a set of images such as the given image database. Formulating this transductive problem in a probabilistic framework, the proposed algorithm, Discriminant-EM (D-EM), not only estimates the parameters of a generative model, but also finds a linear transformation to relax the assumption of pro...

In this paper, we propose a novel transductive learning framework named manifold-ranking based image retrieval (MRBIR). Given a query image, MRBIR first makes use of a manifold ranking algorithm to explore the relationship among all the data points in the feature space, and then measures relevance between the query and all the images in the database accordingly, which is different from traditional similarity metrics based on pair-wise distance. In relevance feedback, if only positive examples are available, they are added to the query set to improve the retrieval result; if examples of both labels can be obtained, MRBIR discriminately spreads the ranking scores of positive and negative examples, considering the asymmetry between these two types of images. Furthermore, three active learning methods are incorporated into MRBIR, which select images in each round of relevance feedback according to different principles, aiming to maximally improve the ranking result. Experimental results on a general-purpose image database show that MRBIR attains a significant improvement over existing systems from all aspects.

By using relevance feedback [6], Content-Based Image Retrieval (CBIR) allows the user to retrieve images interactively. The user can select the most relevant images and provide a weight of preference for each relevant image. The high level concept borne by the user and perception subjectivity of the user can be captured by the system to some degree. This paper proposes an approach to utilize both positive and negative feedbacks for image retrieval. Support Vector Machines (SVM) is applied to classifying the positive and negative images. The SVM learning results are used to update the preference weights for the relevant images. This approach releases the user from manually providing preference weight for each positive example. Experimental results show that the proposed approach has improvement over the previous approach [5] that uses positive examples only. 1.

Unlike traditional database management systems, in content-based multimedia retrieval databases, it is difficult for users to express their exact information need directly in a precise query. A typical interface allows users to express their information need using examples of objects similar to the ones they wish to retrieve. Such a user interface, however, requires mechanisms to learn the query representation from the examples. In this paper, we describe the query refinement framework implemented in the Multimedia Analysis and Retrieval System (MARS) for learning query representations using relevance feedback. The proposed framework uses a query expansion approach towards modifying the query representation in which relevant objects are added to the query. Furthermore, query reweighting techniques are used to adjust similarity functions. 1 Introduction Advances in image processing, database management, and information retrieval (IR) have resulted in content-based multimedia retrieva...