Impact of Feature Selection on Classifier Testing Validity

This paper studies the influence of feature selection (pre-processing stage in data mining) on classifier testing, in particular, when data mining techniques are applied in bioinformatics (classification and pattern recognition task using antibody display data in this case). The study experimentally evaluates classifier testing validity if the data set used in testing has already been used in feature selection in pre-processing because of the possible classification model corruption and adaptation to test data. The experiments employ ten feature selection methods – four subset selection methods (correlation-based, consistency evaluator and two types of wrappers) and six feature ranking methods (Chi-square statistic, Gain Ratio, Information Gain, OneR, ReliefF and SVM), and evaluates four classifiers (C4.5, Random forest, SVM and Naive Bayes) using data sets that were used in feature selection and absolutely independent test sets