Gaussian Process Belief Propagation

The framework of graphical models is a cornerstone of applied Statistics, allowing for an intuitive graphical specification of the main features of a model, and providing a basis for general Bayesian inference computations though belief propagation (BP). In the latter, messages are passed between marginal beliefs of groups of variables. In parametric models, where all variables are of fixed finite dimension, these beliefs and messages can be represented easily in tables or parameters of exponential families, and BP techniques are widely used in this case. In this paper, we are interested in nonparametric models, where belief representations do not have a finite dimension, but grow with the dataset size. In the presence of several dependent domain variables, each of which is represented as a nonparametric random field, we aim for a synthesis of BP and nonparametric approximate inference techniques. We highlight the difficulties in exercising this venture and suggest possible techniques for remedies. We demonstrate our program using the example of semiparametric latent factor models (Teh et.al, 2004), which can be used to model conditional dependencies between multiple responses