Multiple-Instance learning is a way of modeling ambiguity in supervised learning examples. Each example is a bag of instances, but only the bag is labeled - not the individual instances. A bag is labeled negative if all the instances are negative, and positive if at least one of the instances in positive. We apply the Multiple-Instance learning framework to the problem of learning how to classify natural images. Images are inherently ambiguous since they can represent many different things. A user labels an image as positive if the image somehow contains the concept. Each image is a bag, and the instances are various sub-regions in the image. From a small collection of positive and negative examples, we can learn the concept and then use it to retrieve images that contain the concept from a large database. We show that the Diverse Density algorithm performs well in this task, that simple hypothesis classes are sufficient to classify natural images, and that user interaction helps to im...

We suggest the use of the color correlogram as a generic indexing tool to tackle various computer vision problems. Correlograms were shown to be very effective for contentbased image retrieval [4]. We adapt the correlogram to handle the problems of image subregion querying, object localization, object tracking, and cut detection. Experimental results suggest that the color correlogram is much more effective than the histogram for these applications, with insignificant additional computational, storage, or processing cost. We also provide a technique to cut down the storage requirement of correlograms so that it is the same as that of histograms, with only negligible performance penalty compared to the original correlogram. 1

Most of the current image retrieval systems use "one-shot" queries to a database to retrieve similar images. Typically a K-nearest neighbor kind of algorithm is used, where weights measuring feature importance along each input dimension remain fixed (or manually tweaked by the user), in the computation of a given similarity metric. However, the similarity does not vary with equal strength or in the same proportion in all directions in the feature space emanating from the query image. The manual adjustment of these weights is time consuming and exhausting. Moreover, it requires a very sophisticated user. In this paper, we present a novel probabilistic method that enables image retrieval procedures to automatically capture feature relevance based on user's feedback and that is highly adaptive to query locations. Experimental results are presented that demonstrate the efficacy of our technique using both simulated and real-world data.

By using relevance feedback [6], Content-Based Image Retrieval (CBIR) allows the user to retrieve images interactively. The user can select the most relevant images and provide a weight of preference for each relevant image. The high level concept borne by the user and perception subjectivity of the user can be captured by the system to some degree. This paper proposes an approach to utilize both positive and negative feedbacks for image retrieval. Support Vector Machines (SVM) is applied to classifying the positive and negative images. The SVM learning results are used to update the preference weights for the relevant images. This approach releases the user from manually providing preference weight for each positive example. Experimental results show that the proposed approach has improvement over the previous approach [5] that uses positive examples only. 1.

In this paper, we present MediaNet, which is a knowledge representation framework that uses multimedia content for representing semantic and perceptual information. The main components of MediaNet include conceptual entities, which correspond to real word objects, and relationships among concepts. MediaNet allows the concepts and relationships to be defined or exemplified by multimedia content such as images, video, audio, graphics, and text. MediaNet models the traditional relationship types such as generalization and aggregation but adds additional functionality by modeling perceptual relationships based on feature similarity. For example, MediaNet allows a concept such as "car" to be deftned as a type of a "transportation vehicle", but which is further defined and illustrated through example images, videos and sounds of cars. In constructing the MediaNet framework, we have built on the basic principles of semiotics and semantic networks in addition to utilizing the audio-visual content description framework being developed as part of the MPEG-7 multimedia content description standard.

Relevance feedback [1] has been a powerful tool for interactive Content-Based Image Retrieval (CBIR). During the retrieval process, the user selects the most relevant images and provides a weight of preference for each relevant image. User's high level query and perception subjectivity can be captured to some extent by dynamically updated low-level feature weights based on the user's feedback. However, in MARS [2] only the positive feedbacks, i.e., relevant images are considered. In this paper, a novel approach is proposed by providing both positive and negative feedbacks for Support Vector Machines (SVM) learning. The SVM learning results are used to update the weights of preference for relevant images. Priorities are given to the positive feedbacks that have larger distances to the hyperplane determined by the support vectors. This approach releases the user from manually providing preference weight for each positive example, i.e., relevant image as before. Experimental results show ...

Most of the current image retrieval systems use "one-shot" queries to a database to retrieve similar images. Typically a K-NN (nearest neighbor) kind of algorithm is used where the weights of the features that are used to represent images remain fixed (or manually tweaked by the user) in the computation of a given similarity metric. However, neither all of the features are equally important for a given query nor a similarity metric is optimal for all kinds of images in a database. The manual adjustment of these weights and the selection of similarity metric are exhausting. Moreover, they require a very sophisticated user. In this paper we present a novel image retrieval system that continuously learns the weights of features and selects an appropriate similarity metric based on the user's feedback given as positive or negative image examples. Experimental results are presented that provide the objective evaluation of learning behavior of the system for image retrieval.

this article with a review of the issues in content-based visual query, then describe the current MetaSeek implementation. We present the results of experiments that evaluated the implementation in comparison to a previous version of the system and a baseline engine that randomly selects the individual search engines to query. We conclude by summarizing open issues for future research.

In the IBM CueVideo TM project we study various aspects of fully automated video indexing, browsing and retrieval. The technical aspects include audio processing, speech recognition, image processing and information retrieval. Equally important, however, is exploring user expectations and conducting user studies. We focus on the field of video for Training and Education, including Distributed Learning, Remote Education, and Just-in-Time Learning. This paper describes the use of

In this paper, we examine emerging frontiers in the evolution of content-based retrieval systems that rely on an intelligent infrastructure. Here, we refer to intelligence as the capabilities of the systems to build and maintain situational or world models, utilize dynamic knowledge representations, exploit context, and leverage advanced reasoning and learning capabilities. We argue that these elements are essential to producing effective systems for retrieving audio-visual content at semantic levels matching those of human perception and cognition. In this paper, we review relevant research on the understanding of human intelligence and construction of intelligent systems in the fields of cognitive psychology, artificial intelligence, semiotics, and computer vision. We also discuss how some of the principal ideas from these fields lead to new opportunities and capabilities for content-based retrieval systems. Finally, we describe some of our efforts in these directions. In particular, we present MediaNet, a multimedia knowledge presentation framework, and some MPEG-7 description tools that facilitate and enable intelligent content-based retrieval.