Mutual Information Based on Renyi’s Entropy Feature Selection

Abstract—Feature selection problem has become the focus of much pattern classification research and mutual information is more and more important in the feature selection algorithms. We proposed normalized mutual information based on Renyi’s quadratic entropy feature selection, which reduces the computational complexity, relying on the efficient estimation of the mutual information. Then we combine NMIFS with wrappers into a two-stage feature selection algorithm. This helps us find more charactering feature subset. We perform some experiments to compare the efficiency and classification accuracy to other MI-based feature selection algorithm. Results show that our method leads to promising improvement on computation complexity. Keywords-feature selection; mutual information; Renyi Entropy; estimation of entropy; NMIFS I