A note on the applied use of MDL approximations

An applied problem is discussed in which two nested psychological models of retention are compared using Minimum Description Length (MDL). The standard Fisher information approximation (FIA) to the Normalized Maximum Likelihood (NML) is calculated for these two models, with the result that the full model is assigned a smaller complexity, even for moderately large samples. A geometric interpretation for this behavior is considered, along with its practical implications. The Minimum Description Length (MDL) principle (Rissanen 1978; see also Grünwald 1998) has attracted interest in applied fields, because it allows comparisons between non-nested and misspecified models without requiring restrictive assumptions. Of particular interest is the Normalized Maximum Likelihood (NML) criterion, NML = p(X |θ∗(X))∫ p(Y |θ∗(Y)) dY where θ∗(X) denotes the maximum likelihood estimate (MLE) for the data X. While the NML has many desirable properties (Rissanen 2001), a practical difficulty is that the normalization term is difficult to evaluate. Consequently, an approach based on Fisher information (FIA) is often used in its place (e.g. Rissanen 1996). This criterion is given by, FIA = − ln p(X |θ∗(X)) + k 2 l