Multivariate maximum entropy identification, transformation, and dependence

This paper shows that multivariate distributions can be characterized as maximum entropy (ME) models based on the well-known general representation of density function of the ME distribution subject to moment constraints. In this approach, the problem of ME characterization simplifies to the problem of representing the multivariate density in the ME form, hence there is no need for case-by-case proofs by calculus of variations or other methods. The main vehicle for this ME characterization approach is the information distinguishability relationship, which extends to the multivariate case. Results are also formulated that encapsulate implications of the multiplication rule of probability and the entropy transformation formula for ME characterization. The dependence structure of multivariate ME distribution in terms of the moments and the support of distribution is studied. The relationships of ME distributions with the exponential family and with bivariate distributions having exponential family conditionals are explored. Applications include new ME characterizations of many bivariate distributions, including some singular distributions.Bivariate distribution Characterization Dependence Exponential family Kullback-Leibler information Singular distribution