Towards the personalization of algorithms evaluation in data mining

Like model selection in statistics, the choice of appropriate Data Mining Algorithms (DM-Algorithms) is a very important task in the process of Knowledge Discovery. Due to this fact it is necessary to have sophisticated metrics that can be used as comparators to evaluate alternative DM-algorithms. It has been shown in literature, that Data Envelopment Analysis (DEA) is an appropriate platform to develop multi-criteria evaluation metrics that can consider-in contrary to mono-criteria metrics- all positive and negative properties of DM-algorithms. We discuss different extensions of DEA that enable consideration of qualitative properties of DM-algorithms and consideration of users preferences in development of evaluation metrics. The results open new discussions in the general debate on model selection in statistics and machine learning. 1