We present a statistical model for organizing image collections which integrates semantic information provided by associated text and visual information provided by image features. The model is very promising for information retrieval tasks such as database browsing and searching for images based on text and/or image features. Furthermore, since the model learns relationships between text and image features, it can be used for novel applications such as associating words with pictures, and unsupervised learning for object recognition. 1.

Extending beyond the boundaries of science, art, and culture, content-based multimedia information retrieval provides new paradigms and methods for searching through the myriad variety of media all over the world. This survey reviews 100+ recent articles on content-based multimedia information retrieval and discusses their role in current research directions which include browsing and search paradigms, user studies, affective computing, learning, semantic queries, new features and media types, high performance indexing, and evaluation techniques. Based on the current state of the art, we discuss the major challenges for the future.

Abstract not found

The rapid development of web image search engines has enabled users to search hundred million of images available on the Web. However, due to the unsatisfactory performance of current search technologies, people still need to spend much time in navigating through the large number of result pages to find images of their interest. In this paper, we analyze the user information needs for web image search results browsing and propose to employ a similarity-based organization to present the search results. A user study is carried out to compare our approach with a ranking-based list interface and a cluster-based interface. Experimental results show that similarity-based presentation can help users to explore image search results more naturally and efficiently.

In this paper, we propose a novel desig ofcontent access and extractionalgracti for compressedimag browsing and indexing which is critical for all visual information systems. Byanalyzing the relationship between DCT coe#cients of one block of8 8 pixels and its four sub-blocks of 4 4 pixels, the proposed algosedf extract an approximatedimag with smaller size forindexing and contentbrowsing withoutincurring full decompression. While thecomputing cost issig###fb#k8 lower than full decompression, the approximatedimag also reserves the content features, which are su#cient forindexing and browsing as evidenced by our extensive experiments. ? 2002 PatternRecogf#B4- Society. Published by Elsevier Science Ltd. Allrig4B reserved.

In Content-Based Image Retrieval (CBIR) two querymethods exist: query-by-example and query-by-memory. The user either selects an example image or selects image features retrieved from memory (such as color, texture, spatial attributes, and shape) to define his query. Hitherto, research on CBIR interfaces was absent. Hence, a usability evaluation of existing (CBIR) interfaces was done. Additionally, a study concerning human color cognition was conducted. Based on the resulting findings, a prototype color selection interface was developed. In a theoretical experiment, using Fitts ’ law, was proven that the prototype interface is highly efficient for CBIR purposes. This all resulted in a first set of design guidelines for CBIR color selection interfaces.

With over 800 million pages covering most areas of human endeavor, the World Wide Web is fertile ground for information retrieval. Numerous search technologies have been applied to Web searches, and the dominant search method has yet to be identified. This chapter provides an overview of existing Web search technologies and classifies them into six categories: (i) hyperlink exploration, (ii) information retrieval, (iii) metasearches, (iv) SQL approaches, (v) content-based multimedia searches, and (vi) others. A comparative study of some major commercial and experimental search services is presented, and some future research directions for Web searches are suggested.

A group of research projects based at HP-Labs Bristol, the University of Bristol and ARKive (a new large multimedia database project focused on the worlds biodiversity based in the UK) are working to develop a flexible model for the indexing of multimedia collections that allows users to `annotate' content utilising extensible controlled vocabularies. As part of the educationally focused ARKive-ERA project a series of models for user `annotation' have been developed. The need for these types of user support and tools was identified while conducting pre-design user studies with specialist user groups. The needs centre around the limitations of current online museum and library systems that do not provide support for users to annotate or `tag' multimedia objects of relevance to their particular `community of interest' or with specialised indexing terms. This would enable specialised resource discovery and knowledge sharing with other members of their communities. One example is that of University Lecturers and Researchers studying a particular type of animal behaviour. They may wish to identify all relevant images or video of that particular behaviour and annotate them as good illustrations of aspects of that behaviour. However significant issues arise over for example, the validation of information, access control and the use of such annotations by the resource discovery tools. The paper explores these and other issues and problems involved and how the various models can help provide solutions to the key problems and thus meet the needs of a diverse range of `communities of interest' and adding significant value to on-line multimedia collections.

Recognition of objects in video can offer significant benefits to video retrieval including automatic annotation and content based queries based on the object characteristics. This paper describes our preliminary work toward recognizing objects in video sequences and gives a brief survey of the relevant research in the literature. We use the Kalman filter to obtain segmented blobs from the video, classify the blobs using the probability ratio test, and apply several different temporal filtering methods, which results in sequential classification methods over the video sequence containing the blob. Results from real video sequences are shown.

In the previous signature-based approaches to retrieve symbolic pictures from a large iconic image database, only pictures of type-2 similarity are considered, where a picture A is of type-2 similarity with picture B when both pictures have the same spatial relationship between any two objects in the pictures. When queries of objects, type-0 similarity and type-1 similarity are asked, signatures for objects, type-0 similarity and type-1 similarity are required. Note that a picture is of type-0 similarity when all the spatial category relationships of each object pair are the same to the query picture, and a picture is of type-1 similarity when the picture is of type-0 similarity and all the orthogonal relationships of each object pair are the same to the query picture. Although these four kinds of signatures can be constructed and stored in advance to speed up answering such kinds of queries as in the approach of Lee et al. [1], such a large signature file containing those four kinds of signatures really wastes space. In this paper, we first present the revised version of Y. I. Chang and Yang’s bit-string-based access strategy [2], which constructs type-2 signatures. Based on the revised version, we then propose algorithms to dynamically convert those type-2 signatures into object signatures, type-0