f-entropies, probability of error, and feature selection

The ƒ-entropy family of information measures: u(P1 ,…, pm) = Σf(pk), ƒ concave (e.g., Shannon (1948)Bell Syst. Tech. J. 27, 379–423, 623–656; Suadratic; Daroczy (1970)Inform. Contr. 16, 36–51; etc.), is considered. Characterization of the tightest upper and lower bounds on ƒ-entropies by means of the probability of error, is presented. These bounds are used to derive the dual bounds, i.e., the tightest lower and upper bounds on the probability of error by means of ƒ-entropies. Concerning the use of ƒ-entropies as a tool for feature selection, it is proved that none of the members of this family induce over an arbitrary set of features the same preference order as does the probability of error rule