Integration of articulatory dynamic parameters in HMM/BN based speech recognition system

In this paper, we describe several approaches to integra-tion of the articulatory dynamic parameters along with ar-ticulatory position data into a HMM/BN model based au-tomatic speech recognition system. This work is a contin-uation of our previous study, where we have successfully combined speech acoustic features in form of MFCC with articulatory position observations. Articulatory dynamic parameters are represented by velocity and acceleration coefficients calculated as first and second derivatives of the articulatory position data. All these features are in-tegrated using the HMM/BN acoustic model where each feature corresponds to different Bayesian Network vari-able. By changing the BN topology we can change the way articulatory and acoustic parameters are combined. The evaluation experiments showed that the effect of the articulatory dynamic features greatly depends on the BN structure and that careful data analysis is essential in gain-ing knowledge about the underlying dependencies be-tween different information sources. In comparison with conventional HMM system trained on acoustic data only, the HMM/BN system achieved significant improvement of the recognition performance. 1