EDML for Learning Parameters in Directed and Undirected Graphical Models

EDML is a recently proposed algorithm for learning parameters in Bayesian networks. It was originally derived in terms of approxi-mate inference on a meta-network which un-derlies the Bayesian approach to parame-ter estimation. While this initial derivation helped discover EDML in the first place and provided a concrete context for identifying some of its properties (e.g., in contrast to EM), the formal setting was somewhat te-dious in the number of concepts it drew on. In this paper, we propose a greatly simpli-fied perspective on EDML which casts it as a general approach to continuous optimiza-tion. The new perspective has several advan-tages. First, it makes immediate some re-sults that were non-trivial to prove initially. Second, it facilitates the design of EDML al-gorithms for new graphical models, leading to a new algorithm for learning parameters in Markov networks. We derive this algo-rithm in this paper, and provide an empirical comparison with a commonly used gradient method, showing that EDML can find better estimates several times faster. 1