Abstract—Whereas video tells a narrative by a composition of shots, current video retrieval methods focus mainly on single shots. In retrieval performance estimation, similar shots in a narrative may result in performance over-estimation. We propose an episode-based version of cross-validation leading up to 14% classification improvement over shot based cross-validation. I.

In this paper we introduce and describe the Multimedia Retrieval Markup Language (MRML). This XML-based markup language is the basis for an open communication protocol for content-based image retrieval systems (CBIRSs). MRML was initially designed as a means of separating CBIR engines from their user interfaces. It is, however, also extensible as the basis for standardised performance evaluation procedures. Such a tool is essential for the formulation and implementation of common benchmarks for CBIR. A common protocol can also bring new dynamics to the CBIR field -- it makes the development of new systems faster and more efficient, and opens the door of the CBIR research field to other disciplines such as Human-Computer Interaction. The MRML specifications, as well as the first MRML-compliant applications, are freely available and are introduced in this paper.

This article describes a framework for distributed multimedia retrieval which permits the connection of compliant user interfaces with a variety of multimedia retrieval engines via an open communication protocol, MRML (Multi Media Retrieval Markup Language). It allows the choice of image collection, feature set and query algorithm during run{time, permitting multiple users to query a system adapted to their needs, using the query paradigm adapted to their problem such as query by example (QBE), browsing queries, or query by annotation. User interaction is implemented over several levels and in diverse ways. Relevance feedback is implemented using positive and negative example images that can be used for a best{match QBE query. In contrast, browsing methods try to approach the searched image by giving overviews of the entire collection and by successive renement. In addition to these query methods, Long term o line learning is implemented. It allows feature preferences per ...

Abstract — In this paper we examine several factors that influence the evaluation of multi-class, continuous activity recognition. Currently, there is no standard metric for evaluating and comparing such systems, although many possible error formulations and performance metrics could be adapted from other domains. In order to make progress toward a standard metric appropriate for evaluating activity recognition, we outline the sources of errors in such systems, present different methods for detecting and labeling these errors, and compare existing metrics with a more nuanced performance visualization. We conclude with a discussion concerning the interpretation of the visualization for comparing recognition systems in different domains. I.

While in the area of relational databases interoperability is ensured by common communication protocols (e.g. ODBC/JDBC using SQL), Content Based Image Retrieval Systems (CBIRS) and other multimedia retrieval systems are lacking both a common query language and a common communication protocol. Besides its obvious short term convenience, interoperability of systems is crucial for the exchange and analysis of user data. In this paper, we present and describe an extensible XML-based query markup language, called MRML (Multimedia Retrieval Markup Language). MRML is primarily designed so as to ensure interoperability between dierent content{based multimedia retrieval systems. Further, MRML allows researchers to preserve their freedom in extending their system as needed. MRML encapsulates multimedia queries in a way that enables multimedia (MM) query languages, MM content descriptions, MM query engines, and MM user interfaces to grow independently from each other, reaching a maximum of in...

In this paper, we address an important yet controversial issue in content-based image retrieval (CBIR): performance evaluation (PE). The goal is to develop an objective and comprehensive protocol for quantitative evaluation and comparison of CBIR performances among different approaches. The protocol consists of two elements: acquisition of appropriate ground truth data, and definition of quantitative performance metrics. A large image data set with human-label ground truths has been built as part of the protocol and made available for public use. An application of this protocol to our MiAlbum system is also presented, which shows suitability of such a protocol.

We propose a general-purpose color-based object model called the Multimodal Neighborhood Signature (MNS) with applications in object recognition and image retrieval. Object modeling is example-based and can cope with many appearance variations due to the image formation/rendering process. The local nature of the color representation facilitates robustness to occlusion and clutter. Unlike other methods, neither segmentation nor edge detection is required and the area of homogeneously colored regions is not used. The algorithm is simple to implement and has low storage requirements. In the reported experiments, eight recognition and two other retrieval methods are reviewed and systematically compared with MNS. Results show good and fast performance under severe scale, viewpoint, occlusion, and background change using a single image for object modeling. Although spatial information was not used and its default internal parameters were used, MNS outperformed most compared methods. c ○ 2002 Elsevier Science (USA) Key Words: multimodal neighborhood signature; color; image retrieval; object recognition; mean shift; mns.

Texture description contributes as one of the most important low-level features in content-based image retrieval. In MPEG-7, homogeneous texture descriptor (HTD), texture browsing descriptor (TBD), and edge histogram descriptor (EHD) have been proposed as texture descriptors. However, no comprehensive evaluation and comparison of these three descriptors have been made. In this paper, we propose a comprehensive evaluation and comparison benchmark for feature descriptors, especially for visual descriptors in MPEG-7. In the proposed benchmark, three texture descriptors in MPEG-7 are evaluated and compared. First, the descriptors are analyzed according to the standard criteria. Second, experiments are implemented on the Brodatz texture image database. Analysis of the experimental results shows that each descriptor has some specific characteristics and performs better than the other two in certain applications. The applicability is also summarized for each descriptor. The survey as well as performance evaluation and comparison in this paper provide several guidelines for using these descriptors in image retrieval and other applications.

A patent always contains some images along with the text. Many text based systems have been developed to search the patent database. In this paper, we describe PATSEEK that is an image based search system for US patent database. The objective is to let the user check the similarity of his query image with the images that exist in US patents. The user can specify a set of key words that must exist in the text of the patents whose images will be searched for similarity. PATSEEK automatically grabs images from the US patent database on the request of the user and represents them through an edge orientation autocorrelogram. L1 and L2 distance measures are used to compute the distance between the images. A recall rate of 100 % for 61 % of query images and an average 32 % recall rate for rest of the images has been observed.

Strongly similar subimages contain different views of the same object. In subimage search, the user selects an image region and the retrieval system attempts to find matching subimages in an image database that are strongly similar. Solutions have been proposed using salient features or “interest points ” that have associated descriptor vectors. However, searching large image databases by exhaustive comparison of interest point descriptors is not feasible. To solve this problem, we propose a novel off-line indexing scheme based on the most significant bits (MSBs) of these descriptors. On-line search uses this index file to limit the search to interest points whose descriptors have the same MSB value, a process up to three orders of magnitude faster than exhaustive search. It is also incremental, since the index file for a union of a group of images can be created by merging the index files of the individual image groups. The effectiveness of the approach is demonstrated experimentally on a variety of image databases. 1