Hierarchical correlation compensation for hidden Markov models

In this paper, we present a Hierarchical Correlation Compensation (HCC) scheme to reliably estimate full covariance matrices for Gaussian components in CDHMMs for speech recognition. First, we build a hierarchical tree in the covariance space, where each leaf node represents a Gaussian component in the CDHMM set. For all lower-level nodes in the tree, we estimate a diagonal covariance matrix as usual. But we estimate full matrices for all upper-level nodes since they have large amount of data. For each Gaussian in a leaf node (with diagonal components estimated already), we compensate its offdiagonal components by using a linear combination of a set of prototype covariance matrices, which includes the estimated covariance matrices of all nodes in the tree along the upward path from the leaf all the way to the root. At last, the linear combination weights are estimated based on the maximum likelihood (ML) criterion. We have evaluated the HCC on the DARPA Resource Management (RM) task and an in-house large-vocabulary Chinese dictation task. We have achieved significant error reduction over the best diagonal covariance models. Experimental results also show that HCC yields better performance than other full covariance modeling schemes. 1