DMCA
Look Who's Talking: Speaker Detection Using Video And Audio Correlation (2000)
| Venue: | in IEEE International Conference on Multimedia and Expo |
| Citations: | 55 - 1 self |
BibTeX
@INPROCEEDINGS{Cutler00lookwho's,
author = {Ross Cutler and Larry Davis},
title = {Look Who's Talking: Speaker Detection Using Video And Audio Correlation},
booktitle = {in IEEE International Conference on Multimedia and Expo},
year = {2000},
pages = {1589--1592}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
The visual motion of the mouth and the corresponding audio data generated when a person speaks are highly correlated. This fact has been exploited for lip/speechreading and for improving speech recognition. We describe a method of automatically detecting a talking person (both spatially and temporally) using video and audio data from a single microphone. The audio-visual correlation is learned using a time delayed neural network, which is then used to perform a spatio-temporal search for a speaking person. Applications include video conferencing, video indexing, and improving human computer interaction (HCI). An example HCI application is provided. 1. INTRODUCTION The visual motion of a speaker's mouth is highly correlated with the audio data generated from the voicebox and mouth [5]. This fact has been exploited for lip/speechreading (e.g, [14] [10]) and for combined audio-visual speech recognition (e.g., [3]). We utilize this correlation to detect speakers using video and audio inp...
Keyphrases
speaker detection using video audio correlation visual motion lip speechreading audio data human computer interaction speech recognition audio-visual correlation corresponding audio data single microphone audio inp video conferencing spatio-temporal search neural network person speaks speaking person audio-visual speech recognition video indexing example hci application
