Prosody-driven head-gesture animation

We present a new framework for joint analysis of head gesture and speech prosody patterns of a speaker towards automatic realistic syn-thesis of head gestures from speech prosody. The proposed two-stage analysis aims to “learn ” both elementary prosody and head ges-ture patterns for a particular speaker, as well as the correlations be-tween these head gesture and prosody patterns from a training video sequence. The resulting audio-visual mapping model is then em-ployed to synthesize natural head gestures from arbitrary input test speech given a head model for the speaker. Objective and subjective evaluations indicate that the proposed synthesis by analysis scheme provides natural looking head gestures for the speaker with any input test speech. Index Terms — Man-machine systems, multimedia systems, ges-ture and prosody analysis, gesture synthesi