DMCA
Adding semantics to detectors for video retrieval (2007)
Cached
Download Links
| Venue: | IEEE Transactions on Multimedia |
| Citations: | 77 - 14 self |
BibTeX
@ARTICLE{Snoek07addingsemantics,
author = {Cees G. M. Snoek and Bouke Huurnink and Laura Hollink and Maarten De Rijke and Guus Schreiber and Marcel Worring},
title = {Adding semantics to detectors for video retrieval},
journal = {IEEE Transactions on Multimedia},
year = {2007},
volume = {9}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract — In this paper, we propose an automatic video retrieval method based on high-level concept detectors. Research in video analysis has reached the point where over 100 concept detectors can be learned in a generic fashion, albeit with mixed performance. Such a set of detectors is very small still compared to ontologies aiming to capture the full vocabulary a user has. We aim to throw a bridge between the two fields by building a multimedia thesaurus, i.e. a set of machine learned concept detectors that is enriched with semantic descriptions and semantic structure obtained from WordNet. Given a multimodal user query, we identify three strategies to select a relevant detector from this thesaurus, namely: text matching, ontology querying, and semantic visual querying. We evaluate the methods against the automatic search task of the TRECVID 2005 video retrieval benchmark, using a news video archive of 85 hours in combination with a thesaurus of 363 machine learned concept detectors. We assess the influence of thesaurus size on video search performance, evaluate and compare the multimodal selection strategies for concept detectors, and finally discuss their combined potential using oracle fusion. The set of queries in the TRECVID 2005 corpus is too small to be definite in our conclusions, but the results suggest promising new lines of research. Index Terms — Video retrieval, concept learning, knowledge modeling, content analysis and indexing, multimedia information systems I.
Keyphrases
concept detector video retrieval text matching multimodal user query multimedia information system new line thesaurus size relevant detector oracle fusion semantic description generic fashion multimedia thesaurus semantic visual querying concept learning video analysis high-level concept detector video search performance content analysis news video archive multimodal selection strategy video retrieval benchmark semantic structure knowledge modeling automatic search task mixed performance index term video retrieval ontology querying automatic video retrieval method full vocabulary
