Abstract

Frames from a voice-driven animation, computed from a single baby picture and an adult model of facial control. Note the changes in upper facial expression. See figures 5, 6 and 7 for more examples of predicted mouth shapes. We introduce a method for predicting a control signal from another related signal, and apply it to voice puppetry: Generating full facial animation from expressive information in an audio track. The voice puppet learns a facial control model from computer vision of real facial behavior, automatically incorporating vocal and facial dynamics such as co-articulation. Animation is produced by using audio to drive the model, which induces a probability distribution over the manifold of possible facial motions. We present a lineartime closed-form solution for the most probable trajectory over this manifold. The output is a series of facial control parameters, suitable for driving many different kinds of animation ranging from video-realistic image warps to 3D cartoon characters.

Citations

422	An Inequality and Associated Maximization Technique in Statistical Estimation of a Markov Process - Baum - 1972
189	Video Rewrite: Driving Visual Speech with Audio - Bregler, Covell, et al. - 2002
37	Pattern discovery via entropy minimization - Brand - 1999
17	ªStructure Discovery in Conditional Probability Models via an Entropic Prior and Parameter - Brand - 1999
6	Spoken language processing in the persona conversational assistant - Ball, Ling - 1995
2	Read my lips: Where? How? When? And so... What - Benoit, Abry, et al. - 1995
1	puppetry. Submitted to Int - Shadow - 1999
1	Audio-visual interaction in nultimedia communication - Chen, Rao - 1997