Audio-Visual Probabilistic Tracking of Multiple Speakers (2007)
Cached
Download Links
| Venue: | in Meetings,” IEEE Trans. on Audio, Speech, and Language Processing |
| Citations: | 47 - 4 self |
BibTeX
@INPROCEEDINGS{Gatica-perez07audio-visualprobabilistic,
author = {Daniel Gatica-perez and Guillaume Lathoud and Jean-marc Odobez and Iain Mccowan},
title = {Audio-Visual Probabilistic Tracking of Multiple Speakers},
booktitle = {in Meetings,” IEEE Trans. on Audio, Speech, and Language Processing},
year = {2007},
pages = {601--616}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Tracking speakers in multiparty conversations constitutes a fundamental task for automatic meeting analysis. In this paper, we present a novel probabilistic approach to jointly track the location and speaking activity of multiple speakers in a multisensor meeting room, equipped with a small microphone array and multiple uncalibrated cameras. Our framework is based on a mixed-state dynamic graphical model defined on a multiperson state-space, which includes the explicit definition of a proximity-based interaction model. The model integrates audio-visual (AV) data through a novel observation model. Audio observations are derived from a source localization algorithm. Visual observations are based on models of the shape and spatial structure of human heads. Approximate inference in our model, needed given its complexity, is performed with a Markov Chain Monte Carlo particle filter (MCMC-PF), which results in high sampling efficiency. We present results-based on an objective evaluation procedure- that show that our framework (1) is capable of locating and tracking the position and speaking activity of multiple meeting participants engaged in real conversations with good accuracy; (2) can deal with cases of visual clutter and occlusion; and (3) significantly outperforms a traditional sampling-based approach. I.
Keyphrases
multiple speaker audio-visual probabilistic tracking speaking activity source localization algorithm real conversation proximity-based interaction model mixed-state dynamic graphical model automatic meeting analysis audio observation multisensor meeting room multiple meeting participant visual clutter traditional sampling-based approach spatial structure explicit definition multiperson state-space objective evaluation procedure human head small microphone array multiple uncalibrated camera multiparty conversation high sampling efficiency novel observation model good accuracy novel probabilistic approach fundamental task approximate inference visual observation
