This paper shows how an articulatory model, able to produce acoustic signals from articulatory motion, can learn to speak, i.e. coordinate its movements in such a way that it utters meaningful sequences of sounds belonging to a given language. This complex learning procedure is accomplished in four major steps: (a) a babbling phase, where the device builds up a model of the forward transforms, i.e. the articulatory-to-audio-visual mapping; (b) an imitation stage, where it tries to reproduce a limited set of sound sequences by audio-visual-to-articulatory inversion; (c) a "shaping" stage, where phonemes are associated with the most efficient available sensori-motor representation; and finally, (d) a "rhythmic" phase, where it learns the appropriate coordination of the activations of these sensori-motor targets.

This chapter describes how an artificial device, able to produce acoustic signals from articulatory motion, can learn to speak, i.e. coordinate its articulatory movements in such a way that it utters meaningful sequences of sounds belonging to a given language. This complex learning procedure, accomplished within a few years by the human child, is simulated in four major steps: (a) a babbling phase, where the device builds up a model of the forward kinematics, i.e. the articulatory-to-audio-visual mapping; (b) an imitation stage, where it tries to reproduce a limited set of sound sequences by audio-visual-to-articulatory inversion including a normalization procedure; (c) a "shaping" stage, where phonemes are associated with sensori-motor representation; and finally, (d) a "rhythmic" phase, where it learns the appropriate coordination of the activations of these sensori-motor targets. This artificial device has thus an ear which delivers both the control signals and the identification of pe...

We present here the "labiophone", a virtual system for audio-visual speech communication. A clone of the speaker is animated at distance by articulatory movements extracted from the speaker's image and captured thanks to a video-camera centered on the speaker's face. The clone consists of a mesh driven by a few articulatory parameters and clothed by blended textures. The characteristics of the articulatory model and the textures blending are transmitted at the initiation of the dialog. Then only articulatory parameters are transmitted at a very low bit rate through the telecommunication or web network. Preliminary evaluation of such a system is presented below. Keywords: speech, facial animation, articulatory modelling, movement estimation, texture mapping. 1. INTRODUCTION Speech communication is multi-modal: if auditory and visual perception provide complementary information about the speaker and its emotional state, they collaborate intimately to enhance the intelligibility of the ...