Bayesian Estimation of Latently-grouped Parameters in Undirected Graphical Models

In large-scale applications of undirected graphical models, such as social networks and biological networks, similar patterns occur frequently and give rise to simi-lar parameters. In this situation, it is beneficial to group the parameters for more efficient learning. We show that even when the grouping is unknown, we can in-fer these parameter groups during learning via a Bayesian approach. We impose a Dirichlet process prior on the parameters. Posterior inference usually involves cal-culating intractable terms, and we propose two approximation algorithms, namely a Metropolis-Hastings algorithm with auxiliary variables and a Gibbs sampling al-gorithm with “stripped ” Beta approximation (Gibbs SBA). Simulations show that both algorithms outperform conventional maximum likelihood estimation (MLE). Gibbs SBA’s performance is close to Gibbs sampling with exact likelihood cal-culation. Models learned with Gibbs SBA also generalize better than the models learned by MLE on real-world Senate voting data.