Visual Analysis of Viseme Dynamics

Face to face dialogue is the most natural mode of communication between hu-mans. The combination of human visual perception of expression and perception in changes in intonation provides semantic information that communicates idea, feelings and concepts. The realistic modelling of speech movements, through au-tomatic facial animation, and maintaining audio-visual coherence is still a chal-lenge in both the computer graphics and film industry. A common approach to producing visual speech is to interpolate parameters that describe mouth varia-tion in sequence, known as visemes. A viseme corresponds to a phoneme in an utterance. Most talking head systems use sets of static visemes, represented by a single mouth shape image or 3D model. However, discretising visemes in this way does not account for context-dependent dynamic information, coarticulation. This thesis presents several visual analysis and dynamic modelling techniques for visual phones. This spans several areas of work, from capture and representation through to analysis and synthesis of speech movements and coarticulation.