Abstract

A system for rule based audiovisual text-to-speech synthesis has been created. The system is based on the KTH text-to-speech system which has been complemented with a three-dimensional parameterized model of a human face. The face can be animated in real time, synchronized with the auditory speech. The facial model is controlled by the same synthesis software as the auditory speech synthesizer. A set of rules that takes coarticulation into account has been developed. The audiovisual text-to-speech system has also been incorporated into a spoken man-machine dialogue system that is being developed at the department. 1. INTRODUCTION The visual channel in speech communication is of great importance, as has been demonstrated by for example McGurk [6]. A view of the face can improve intelligibility of both natural and synthetic speech, especially under degraded acoustic conditions [5]. Moreover, visual signals can express emotion, add emphasis to the speech and support the interaction in a...

Citations

245	Hearing lips and seeing voices - McGurk, MacDonald - 1976
172	Modeling Coarticulation in Synthetic Visual Speech - Cohen, Massaro - 1993
140	A parameterized model for facial animation - Parke - 1982
32	The Waxholm system - a progress report - Bertenstam, Carlson, et al. - 1995
25	A multi-language text-to-speech module - Carlson, Granström, et al. - 1982
25	Communication and Coarticulation in Facial Animation - Pelachaud - 1991
21	Real-time analysis-synthesis and intelligibility of talking faces - LeGoff, Guiard-Marigny, et al. - 1994
16	Experiments with voice modelling in speech synthesis, Speech communication - Carlson, Granström, et al. - 1991