An Introduction to Model Selection: Tools and Algorithms

Model selection is a complicated matter in science, and psychology is no exception. In particular, the high variance in the object of study (i.e., humans) prevents the use of Popper's falsification principle (which is the norm in other sciences). Therefore, the desirability of quantitative psychological models must be assessed by measuring the capacity of the model to fit empirical data. In the present paper, an error measure (likelihood), as well as five methods to compare model fits (the likelihood ratio test, Akaike's information criterion, the Bayesian information criterion, bootstrapping and cross-validation), are presented. The use of each method is illustrated by an example, and the advantages and weaknesses of each method are also discussed. The main goal of scientific investigation is to explain and predict empirical phenomena. The former is usually handled by formulating theories which explain the observations using one or several abstract concepts that are causally related to the experience (Humes, 1888). However, explaining observed phenomena is not sufficient: the tentative explanation, expressed as a particular theory, must also account for new observations (prediction). This is often achieved by operationalizing the theory into a model. A model is a specification of a theory 1 which makes the prediction of new phenomena possible. As a result, the predictions of a given model can be empirically tested in order to assess its desirability. In particular, Karl Popper argued in favour of a simple way to determine the scientifi