u.ac.jp

As one of the techniques for robust speech recognition under noisy environments, audio-visual speech recognition (AVSR) using lip dynamic scene information together with audio in-formation is attracting attention, and the research has ad-vanced in recent years. However, in visual speech recogni-tion (VSR), when a face turns sideways, the shape of the lip as viewed from the camera changes and the recognition accu-racy degrades significantly. Therefore, many of the conven-tional VSR methods are limited to situations in which the face is viewed from the front. This paper proposes a VSR method to convert faces viewed from various directions into faces that are viewed from the front using Active Appear-ance Models (AAM). In the experiment, even when the face direction changes about 30 degrees relative to a frontal view, the recognition accuracy improved significantly