Estimating Dependency Structures for non-Gaussian Components with Linear and Energy Correlations

The statistical dependencies which indepen-dent component analysis (ICA) cannot re-move often provide rich information beyond the ICA components. It would be very useful to estimate the dependency structure from data. However, most models have concen-trated on higher-order correlations such as energy correlations, neglecting linear correla-tions. Linear correlations might be a strong and informative form of a dependency for some real data sets, but they are usually com-pletely removed by ICA and related meth-ods, and not analyzed at all. In this pa-per, we propose a probabilistic model of non-Gaussian components which are allowed to have both linear and energy correlations. The dependency structure of the components is explicitly parametrized by a parameter ma-trix, which denes an undirected graphical model over the latent components. Further-more, the estimation of the parameter matrix is shown to be particularly simple because using score matching, the objective function is a quadratic form. Using articial data, we demonstrate that the proposed method is able to estimate non-Gaussian components and their dependency structures, as it is de-signed to do. When applied to natural images and outputs of simulated complex cells in the primary visual cortex, novel dependencies be-tween the estimated features are discovered