Efficient algorithms to improve feature selection accuracy in high dimensional data

© 2020 Batugahage Kushani Anuradha PereraFeature selection plays a vital role in machine learning by removing the features which are irrelevant for the learning task in high dimensional datasets. Selecting the relevant features before the learning task often improves the prediction accuracy of the learning models, reduces the training and prediction times of the learning models and results in simple learning models. However, selecting the optimal feature subset is an NP-hard problem and achieving high prediction accuracy with low computational costs remains a challenge. In this thesis, we develop efficient feature selection algorithms which gain high prediction accuracy in machine learning tasks to address this problem. First, we propose a method to improve the feature selection accuracy in multi-class binary data compared to state-of-the-art methods for binary feature selection. Despite many feature selection methods specifically designed for different data types, only a few methods are specifically designed for binary data. These few methods also result in poor class separation; therefore, low prediction accuracy in multi-class binary datasets. In our feature selection method, we first propose two new feature selection measures which exploit the special properties in binary data. Second, we propose a feature selection algorithm, which uses these measures to achieve good class separation in multi-class binary data. We experimentally show that compared to the existing feature selection methods, the proposed method achieves high classification accuracy in multi-class binary datasets with reasonable computational costs. The proposed method has wide applicability in domains such as image and gene analysis and text classification in which binary datasets are commonly used. Second, we propose a framework, which facilitates the supervised filter feature selection algorithms to exploit feature group information from external sources of knowledge. Among the various feature selection methods available, filter methods use statistical measures, and wrapper and embedded methods use the classification accuracy of a learning model to evaluate the features. Compared to the wrapper and embedded methods, filter methods are advantageous in terms of the interpretability of the results, simplicity and computational efficiency. However, in the literature, feature group information, obtained from external sources of knowledge, is rarely used to improve the filter feature selection approach. We theoretically and experimentally show that using the proposed framework, external feature group information can be incorporated into some filter feature selection algorithms with no additional computational costs. We also show that the new algorithm which exploits feature group information achieves higher classification accuracy than the original algorithm and other existing feature selection methods. Third, we propose a framework to incorporate feature group information from the external sources of knowledge into unsupervised feature selection methods. Due to the unavailability of the class labels, unsupervised feature selection is more challenging than supervised feature selection, yet has wide applicability in real world applications. We experimentally show that the proposed framework can be used to develop new unsupervised feature selection algorithms which achieve high clustering performance with low computational costs. As the current framework is limited to disjoint feature groups, in our fourth contribution, we extend it to exploit pairwise feature correlations as well, resulting in a more generic framework. Proposed group based and pairwise correlation based feature selection methods have wide applicability in areas such as genomic, text and image data analysis which involve datasets whose features show group behaviours and pairwise correlations