Abstract—This paper presents the semantic pathfinder architecture for generic indexing of multimedia archives. The semantic pathfinder extracts semantic concepts from video by exploring different paths through three consecutive analysis steps, which we derive from the observation that produced video is the result of an authoring-driven process. We exploit this authoring metaphor for machine-driven understanding. The pathfinder starts with the content analysis step. In this analysis step, we follow a data-driven approach of indexing semantics. The style analysis step is the second analysis step. Here, we tackle the indexing problem by viewing a video from the perspective of production. Finally, in the context analysis step, we view semantics in context. The virtue of the semantic pathfinder is its ability to learn the best path of analysis steps on a per-concept basis. To show the generality of this novel indexing approach, we develop detectors for a lexicon of 32 concepts and we evaluate the semantic pathfinder against the 2004 NIST TRECVID video retrieval benchmark, using a news archive of 64 hours. Top ranking performance in the semantic concept detection task indicates the merit of the semantic pathfinder for generic indexing of multimedia archives. Index Terms—Video analysis, concept learning, benchmarking, content analysis and indexing, multimedia information systems, pattern recognition. 1

This year the UvA-MediaMill team participated in the Feature Extraction and Search Task. We developed a generic approach for semantic concept classification using the semantic value chain. The semantic value chain extracts concepts from video documents based on three consecutive analysis links, named the content link, the style link, and the context link. Various experiments within the analysis links were performed, showing amongst others the merit of processing beyond key frames, the value of style elements, and the importance of learning semantic context. For all experiments a lexicon of 32 concepts was exploited, 10 of which are part of the Feature Extraction Task. Top three system-based ranking in 8 out of the 10 benchmark concepts indicates that our approach is very promising. Apart from this, the lexicon of 32 concepts proved very useful in an interactive search scenario with our semantic video search engine, where we obtained the highest mean average precision of all participants. 1

Keywords: In a few years, access to the content of multimedia data will be a problem of phenomenal proportions, as digital cameras may produce high data rates, and multimedia archives steadily run into petabytes of storage space. As a consequence, in the field of large-scale distributed Multimedia Content Analysis (MMCA) there is an urgent need for the coordinated handling of vast amounts of distributed and replicated data collections. Any sustainable solution to the management problem of distributed (multimedia) data in Grids has to follow a layered approach, starting from a new generation of network controllers and, through a novel middleware, up to an interface to applications. In this paper we report on our first steps of our bigger plan in this direction. Specifically, we introduce a new transport protocol layer (called HI-TP), which is capable of taking advantage of the storage capabilities of network devices, such as routers. The paper presents the protocol definition, and discusses initial performance results for basic transmission functionality. Results indicate that our protocol is capable of achieving higher throughput than traditional transport protocols, even in networks with significant latency.

Abstract not found