Model Selection

Model selection has become an ubiquitous statistical activity in the last decades, none the least due to the computational ease with which many statis-tical models can be fitted to data with the help of modern computing equip-ment. In this article we provide an introduction to the statistical aspects and implications of model selection and we review the relevant literature. 1.1 A General Formulation When modeling data Y, a researcher often has available a menu of competing candidate models which could be used to describe the data. Let M denote the collection of these candidate models. Each model M, i.e., each element of M, can – from a mathematical point of view – be viewed as a collection of probability distributions for Y implied by the model. That is, M is given by M = {Pη: η ∈ H}, where Pη denotes a probability distribution for Y and H represents the ‘pa-rameter ’ space (which can be different across different models M). The ‘pa-rameter ’ space H need not be finite-dimensional. Often, the ‘parameter ’ η will be partitioned into (η1, η2) where η1 is a finite-dimensional parameter whereas η2 is infinite-dimensional. In case the parameterization is identified, i.e., the map η → Pη is injective on H, we will often not distinguish between M and H and will use them synonymously. The model selection problem is now to select – based on the data Y – a model M ̂ = M̂(Y) in M such that M ̂ is a ‘good ’ model for the data Y. Of course, the sense, in which the selected model should be a ‘good ’ model, needs to be made precise and is a crucial point in the analysis. This is particularly important if – as is usually the case – selecting the model M ̂ is not the fina