DMCA
Multimodal multispeaker probabilistic tracking in meetings (2005)
| Venue: | in Proc. Int. Conf. on Multimodal Interfaces (ICMI |
| Citations: | 17 - 6 self |
BibTeX
@INPROCEEDINGS{Gatica-perez05multimodalmultispeaker,
author = {Daniel Gatica-perez and Guillaume Lathoud and Jean-marc Odobez and Iain Mccowan},
title = {Multimodal multispeaker probabilistic tracking in meetings},
booktitle = {in Proc. Int. Conf. on Multimodal Interfaces (ICMI},
year = {2005},
pages = {183--190}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Tracking speakers in multiparty conversations constitutes a fundamental task for automatic meeting analysis. In this paper, we present a probabilistic approach to jointly track the location and speaking activity of multiple speakers in a multisensor meeting room, equipped with a small microphone array and multiple uncalibrated cameras. Our framework is based on a mixed-state dynamic graphical model defined on a multiperson state-space, which includes the explicit definition of a proximity-based interaction model. The model integrates audio-visual (AV) data through a novel observation model. Audio observations are derived from a source localization algorithm. Visual observations are based on models of the shape and spatial structure of human heads. Approximate inference in our model, needed given its complexity, is performed with a Markov Chain Monte Carlo particle filter (MCMC-PF), which results in high sampling efficiency. We present results-based on an objective evaluation procedurethat show that our framework (1) is capable of locating and tracking the position and speaking activity of multiple meeting participants engaged in real conversations with good accuracy; (2) can deal with cases of visual clutter and partial occlusion; and (3) significantly outperforms a traditional sampling-based approach.
Keyphrases
multimodal multispeaker probabilistic tracking speaking activity multiple meeting participant visual clutter traditional sampling-based approach spatial structure explicit definition multiperson state-space good accuracy human head small microphone array multiple uncalibrated camera probabilistic approach partial occlusion high sampling efficiency novel observation model multiple speaker multiparty conversation objective evaluation fundamental task approximate inference visual observation source localization algorithm real conversation proximity-based interaction model mixed-state dynamic graphical model automatic meeting analysis audio observation multisensor meeting room
