The standard EM learning algorithm for MoG consists of iterating the following steps: 1. Calculate responsibilities:

FSMEM, or free split/merge expectation maximization, is a modication of the SMEM algorithm presented by Ueda et al in [1]. Unlike SMEM, FSMEM can modify the number of clusters dynamically. To compare likelihoods across solutions with different number of clusters, a minimal description length term is used. I will be using the FSMEM algorithm exclusively for optimizing mixture of Gaussians, although the algorithm can be applied equally to other clustering problems. The following symbols will be used: K refers to the number of clusters. k is a dummy index enumerating clusters. N refers to the number of data point. n enumerates data points. D refers to the dimensionality of the space. Vectors in data space will be denoted x (x1 xD). Inner product notation will be used whenever convenient: x Σ x ∑ d1 d2 xd1Σd1d2 x d2 1 Mixture of Gaussians and EM Mixture of Gaussians models a probability density function of the form p( x)