Improved speech recognition using adaptive audio-visual fusion via a stochastic secondary classifier

The adaptive fusion of video and audio is one of the fundamental pursuits of audio visual speech recog-nition (AVSR). In this paper the use of a high di-mensional secondary classijier on the word likelihood scores from both the audio and video modalities is in-vestigated fo r the purposes of adaptive fusion. Results are presented that lie above or equal to the boundary of catastrophic fusion across a number of audio noise levels. 1. INTRODUCTION. Verbal communciation uses cues from both the visual and acoustic modalities to convey messages. AVSR becomes advantageous over conventional audio based speech recognition in noisy conditions due to the com