DMCA
A Real-Time Prototype for Small-Vocabulary Audio-Visual ASR
| Venue: | In ICME (Submitted |
| Citations: | 2 - 2 self |
BibTeX
@INPROCEEDINGS{Connell_areal-time,
author = {J. H. Connell and N. Haas and E. Marcheret and C. Neti and G. Potamianos and S. Velipasalar},
title = {A Real-Time Prototype for Small-Vocabulary Audio-Visual ASR},
booktitle = {In ICME (Submitted},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
We present a prototype for the automatic recognition of audiovisual speech, developed to augment the IBM ViaVoice TM speech recognition system. Frontal face, full frame video is captured through a USB 2.0 interface by means of an inexpensive PC camera, and processed to obtain appearance-based visual features. Subsequently, these are combined with audio features, synchronously extracted from the acoustic signal, using a simple discriminant feature fusion technique. On the average, the required computations utilize approximately 67 % of a Pentium TM 4, 1.8 GHz processor, leaving the remaining resources available to hidden Markov model based speech recognition. Real-time performance is therefore achieved for small-vocabulary tasks, such as connected-digit recognition. In the paper, we discuss the prototype architecture based on the ViaVoice TM engine, the basic algorithms employed, and their necessary modifications to ensure real-time performance and causality of the visual front end processing. We benchmark the resulting system performance on stored videos against prior research experiments, and we report a close match between the two. 1.
Keyphrases
small-vocabulary audio-visual asr real-time prototype real-time performance frontal face speech recognition acoustic signal prototype architecture simple discriminant feature fusion technique audio feature ghz processor prior research experiment appearance-based visual feature stored video inexpensive pc camera resulting system performance audiovisual speech viavoice tm engine basic algorithm small-vocabulary task pentium tm required computation full frame video close match automatic recognition connected-digit recognition visual front end processing necessary modification markov model
