Audio-visual stimuli for the evaluation of speech-enhancing algorithms

The benefit from speech-enhancing algorithms in hearing devices may depend not only on the acoustic envi-ronment, but also on the audio-visual perception of speech, e.g., when lip reading, and on other visual cues. Inparticular, the functioning of speech-enhancing algorithms depends on the motion behavior of the user, whichin turn depends on visual cues. In this presentation we introduce various audio-visual stimuli used for evalua-tion of speech-enhancing algorithms in hearing devices. The stimuli include video recordings of the Oldenburgsentence tests (OLSA), real-time animation of lip movement for virtual animated characters, and complex audio-visual environments. We will discuss the effects of the material on speech perception and motion behavior, andoutline applications of these stimuli. Results show that visual cues provide a benefit in terms of speech recep-tion thresholds in the audio-visual OLSA. This benefit can not be found with the animated lip movement usinga simple vocal tract model. However, it was found that the animations are sufficient to achieve natural motionbehavior. This presentation is related to contributions at this conference of Llorach et al. on details of audio-visual speech test material, and Hendrikse et al. on natural motion behavior and its influence on hearingaid algorithm performance in complex listening conditions