Knowledge intensive learning: Combining qualitative constraints with causal independence for parameter learning in probabilistic models

Abstract. In Bayesian networks, prior knowledge has been used in the form of causal independencies between random variables or as quali-tative constraints such as monotonicities. In this work, we extend and combine the two different ways of providing domain knowledge. We de-rive an algorithm based on gradient descent for estimating the param-eters of a Bayesian network in the presence of causal independencies in the form of Noisy-Or and qualitative constraints such as monotonicities and synergies. Noisy-Or structure can decrease the data requirements by separating the influence of each parent thereby reducing greatly the number of parameters. Qualitative constraints on the other hand, allow for imposing constraints on the parameter space making it possible to learn more accurate parameters from a very small number of data points. Our exhaustive empirical validation conclusively proves that the synergy constrained Noisy-OR leads to more accurate models in the presence of smaller amount of data.