Convergence of the em algorithm for gaussian mixtures with unbalanced mixing coefficients

The speed of convergence of the Expecta-tion Maximization (EM) algorithm for Gaus-sian mixture model fitting is known to be dependent on the amount of overlap among the mixture components. In this paper, we study the impact of mixing coefficients on the convergence of EM. We show that when the mixture components exhibit some over-lap, the convergence of EM becomes slower as the dynamic range among the mixing co-efficients increases. We propose a determin-istic anti-annealing algorithm, that signifi-cantly improves the speed of convergence of EM for such mixtures with unbalanced mix-ing coefficients. The proposed algorithm is compared against other standard optimiza-tion techniques like BFGS, Conjugate Gra-dient, and the traditional EM algorithm. Fi-nally, we propose a similar deterministic anti-annealing based algorithm for the Dirichlet process mixture model and demonstrate its advantages over the conventional variational Bayesian approach. 1