Multimedia content processing through cross-modal association (2003)
Cached
Download Links
- [www.oakland.edu]
- [iielab-secs.secs.oakland.edu]
- DBLP
Other Repositories/Bibliography
| Venue: | In MULTIMEDIA ’03: Proceedings of the eleventh ACM international conference on Multimedia |
| Citations: | 40 - 2 self |
BibTeX
@INPROCEEDINGS{Li03multimediacontent,
author = {Dongge Li},
title = {Multimedia content processing through cross-modal association},
booktitle = {In MULTIMEDIA ’03: Proceedings of the eleventh ACM international conference on Multimedia},
year = {2003},
pages = {604--611},
publisher = {ACM Press}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Multimodal information processing has received considerable attention in recent years. The focus of existing research in this area has been predominantly on the use of fusion technology. In this paper, we suggest that cross-modal association can provide a new set of powerful solutions in this area. We investigate different crossmodal association methods using the linear correlation model. We also introduce a novel method for cross-modal association called Cross-modal Factor Analysis (CFA). Our earlier work on Latent Semantic Indexing (LSI) is extended for applications that use offline supervised training. As a promising research direction and practical application of cross-modal association, cross-modal information retrieval where queries from one modality are used to search for content in another modality using low-level features is then discussed in detail. Different association methods are tested and compared using the proposed cross-modal retrieval system. All these methods achieve significant dimensionality reduction. Among them CFA gives the best retrieval performance. Finally, this paper addresses the use of cross-modal association to detect talking heads. The CFA method achieves 91.1 % detection accuracy, while LSI and Canonical Correlation Analysis (CCA) achieve 66.1 % and 73.9 % accuracy, respectively. As shown by experiments, crossmodal association provides many useful benefits, such as robust noise resistance and effective feature selection. Compared to CCA and LSI, the proposed CFA shows several advantages in analysis performance and feature usage. Its capability in feature selection and noise resistance also makes CFA a promising tool for many multimedia analysis applications.
Keyphrases
cross-modal association multimedia content powerful solution offline supervised training robust noise resistance cfa method retrieval performance canonical correlation analysis linear correlation model considerable attention significant dimensionality reduction detection accuracy multimodal information processing many multimedia analysis application cross-modal information retrieval analysis performance feature usage novel method low-level feature practical application noise resistance latent semantic indexing several advantage cross-modal retrieval system crossmodal association promising tool different association method recent year promising research direction different crossmodal association method feature selection many useful benefit effective feature selection new set fusion technology cross-modal factor analysis
