Animating Lip-Sync Characters

Speech animation is traditionally considered as important but tedious work for most applications, especially when tak-ing lip synchronization (lip-sync) into consideration, because the muscles on the face are complex and interact dynami-cally. Although there are several methods proposed to ease the burden on artists to create facial and speech animation, almost none are fast and efficient. In this paper, we intro-duce a framework for synthesizing lip-sync character speech animation from a given speech sequence and its correspond-ing text. Starting from training the dominated animeme models for each kind of phoneme by learning the anima-tion control signals of the character through an EM-style optimization approach, and further decomposing the dom-inated animeme models to the polynomial-fitted animeme models and corresponding dominance functions while taking coarticulation into account. Finally, given a novel speech sequence and its corresponding text, a lip-sync character speech animation can be synthesized in a very short time with the dominated animeme models. The synthesized lip-sync animation can even preserve exaggerated characteris-tics of the character’s facial geometry. Moreover, since our method can synthesize an acceptable and robust lip-sync animation in almost realtime, it can be used for many ap-plications, such as lip-sync animation prototyping, multilin-gual animation reproduction, avatar speech, mass animation production, etc. 1