Recognition of Distant-Talking Speech based on 3-D Trellis Search using a Microphone Array and Adaptive Beamforming

ROBUST1999: IEEE Workshop on Robust Methods for Speech Recognition in Adverse Conditions, May, 1999, Tampere, Finland.Application of a microphone array is a promising solution for realizing distant talking speech recognition, since it can make use of spatial information of sound sources in the environment. Generally, accurate talker localization is very important for speech recognition using a microphone array. However localization of a moving talker is difficult in noisy reverberant environments. We have proposed a novel speech recognition algorithm which considers multiple talker direction hypotheses simultaneously. The algorithm performs Viterbi search in 3-dimensional trellis space composed of talker directions, input frames, and HMM states. As a result, the direction sequence of the talker and the phoneme sequence of the speech are obtained by finding an optimal combination with the highest likelihood. This paper describes the performance improvements by applying an adaptive beamforming technique, and the extension of the algorithm to recognition of simultaneous multiple talkers based on the N-best search