Variable selection methods: an introduction

In order to develop regression/classification models, QSAR analysis typically uses molecular descriptors as independent variables. The number of molecular descriptors has hugely increased over time and nowadays thousands of descriptors, able to describe different aspects of a molecule, can be calculated by means of dedicated software. However, when modelling a particular property or biological activity, it is reasonable to assume that only a small number of descriptors is actually correlated to the experimental response and is, therefore, relevant for building the mathematical model of interest. As a consequence, a key step is the selection of the optimal subset of variables (i.e. molecular descriptors) for the development of the model. This is precisely the aim of the so-called variable selection methods, which allow to: improve interpretability (simple models); neglect not significant effects, thus reducing noise; increase the model predictive ability; speed up modelling time. During the years different variable selection methods have been proposed, from relatively simple to more recent ones that took inspiration from different scientific fields, like genetics and ethology. Furthermore, some methods able to perform both regression and variable selection simultaneously have recently been proposed. In this tutorial we give a short overview of the main variable selection methods. All Subset Models (ASM) The All Subset Models (ASM) method is the most simple and computationally consuming. It consists in the generation of all the possible combinations of the p variables, from size 1 to p, p being the total number of variables. This method guarantees that the best subset of variables is found, but it’s very computationally consuming, being the total number of combinations of p variables given by: 2 1p As a consequence, the method becomes unsuitable for large numbers of variables. If one is interested in developing simple models, i.e. models comprising a limited number k of variables