Choice of Error Cost Function for Training Unobservable Nodes in Bayesian Networks

In the construction of a Bayesian network f fom observed data, the findamental assumption that the variables starting @om the same parent are conditionally independent can be met by introduction of hidden node [Kwoh and Gillies 1994al. In the paper, we have shown that the conditional probability matrices for the hidden node for a triplet, linking three observed nodes, can be determined by the gradient descent method. As in all operational research problem, the quality of the result depends on the ability to locate a feasible solution for the conditional probabilities. In [Kwoh and Gillies 1995aI we presented a paper detailed the methodologies to estimating the initial values of unobservable variables in Bayesian networks. In this paper, we present the concept of determining the best conditional matrices as an estimation problem. The discrepancies between the observed and predicted values are mapped into a monotonic function where its gradients are used for adjusting the parameters to be estimated. We present our investigation of choosing among various popular error cost functions for training the networks with hidden nodes and determined that both cross-entropy and sum-ofsquared error cost functions work equally well for our implementation. 1