Asymptotically optimal approximation of multidimensional pdf\u27s by lower dimensional pdf\u27s

Probability density functions (pdf\u27s) of high dimensionality are impractical to estimate from real data. For accurate estimation, the dimensionality of the pdf can be at most 5-10. In order to reduce the dimensionality a sufficient statistic is usually employed. When none is available, there is no universal agreement on how to proceed. We show how to construct a high-dimension pdf based on the pdf of a low-dimensional statistic that is closest to the true one in the sense of divergence. The latter criterion asymptotically minimizes the probability of error in a decision rule. An application to feature selection for classification is described. © 2006 IEEE