Journal of Articial Intelligence Research 15 (2001) 91-114 Submitted 2/00; published 08/01 Mean- eld methods for a special class of Belief Networks

The chief aim of this paper is to propose mean-eld approximations for a broad class of Belief networks, of which sigmoid and noisy-or networks can be seen as special cases. The approximations are based on a powerful mean-eld theory suggested by Plefka. We show that Saul, Jaakkola, and Jordan's approach is the rst order approximation in Ple-fka's approach, via a variational derivation. The application of Plefka's theory to belief networks is not computationally tractable. To tackle this problem we propose new approx-imations based on Taylor series. Small scale experiments show that the proposed schemes are attractive. 1