Tensor decompositions and sparse log-linear models. arXiv preprint arXiv:1404.0396

Abstract: Contingency table analysis routinely relies on log linear mod-els, with latent structure analysis providing a common alternative. Latent structure models lead to a low rank tensor factorization of the probabil-ity mass function for multivariate categorical data, while log linear models achieve dimensionality reduction through sparsity. Little is known about the relationship between these notions of dimensionality reduction in the two paradigms. We derive several results relating the support of a log-linear model to the nonnegative rank of the associated probability tensor. Moti-vated by these findings, we propose a new collapsed Tucker class of tensor decompositions, which bridge existing PARAFAC and Tucker decomposi-tions, providing a more flexible framework for parsimoniously characteriz-ing multivariate categorical data. Taking a Bayesian approach to inference, we illustrate advantages of the new decompositions in simulations and an application to functional disability data