Age-layered expectation maximization for parameter learning in bayesian networks

The expectation maximization (EM) algo-rithm is a popular algorithm for parame-ter estimation in models with hidden vari-ables. However, the algorithm has several non-trivial limitations, a significant one be-ing variation in eventual solutions found, due to convergence to local optima. Several techniques have been proposed to allay this problem, for example initializing EM from multiple random starting points and select-ing the highest likelihood out of all runs. In this work, we a) show that this method can be very expensive computationally for difficult Bayesian networks, and b) in re-sponse we propose an age-layered EM ap-proach (ALEM) that efficiently discards less promising runs well before convergence. Our experiments show a significant reduction in the number of iterations, typically two- to four-fold, with minimal or no reduction in solution quality, indicating the potential for ALEM to streamline parameter estimation in Bayesian networks