ANALYSIS AND MODELING OF AFFECTIVE AUDIO VISUAL SPEECH BASED ON PAD EMOTION SPACE

This paper analyzes acoustic and visual features for affective audio-visual speech based on PAD (Pleasure-Arousal-Dominance) emotion space. The selected acoustic features include F0 maximum, F0 minimum, duration and energy. A set of Partial Expression Parameters (PEP) is proposed as visual features to describe affective facial movement on talking face. This paper explores the connection between PAD emotion space and acoustic/visual features respectively. The variation of acoustic features is predicted by PAD values, and a PAD-PEP mapping function for facial expression synthesis is built. Experimental result shows that PAD could be properly applied in describing emotional state as well as predicting the acoustic/visual features for affective audio-visual speech synthesis. Index Terms: emotion, acoustic features, facial expression, talking face, speech synthesi