Towards robust feature selection techniques

Robustness of feature selection techniques is a topic of recent interest, especially in high dimensional domains with small sam-ple sizes, where selected feature subsets are subsequently analysed by domain experts to gain more insight into the problem modelled. In this work, we investigate the robustness of various feature selection techniques, and provide a general scheme to improve robust-ness using ensemble feature selection. We show that ensemble feature selection tech-niques show great promise for small sample domains, and provide more robust feature subsets than a single feature selection tech-nique. In addition, we also investigate the ef-fect of ensemble feature selection techniques on classification performance, giving rise to a new model selection strategy. 1