Check it out! A Monte Carlo analysis of the performance of selection criteria and tests for models fitted to Choice Experiments data

When analyzing choice experiment data, the practitioner has many options in the choice of estimators but scarce guidance about how to select the best one. Selecting models frequently used in the choice experiments (CE) literature, we set a Monte Carlo analysis to test the performance of some selection criteria and tests. Usually, a Conditional Logit model (CL) is the first choice for experimental data; afterwards more complex models, like the Random Parameters Logit (RPL) and/or a Latent Class (LC) models are estimated in order to account for heterogeneity of preferences. Since the CL is nested within the RPL, we use the Likelihood Ratio (LR) test to select between the two models. Instead, the choice between LC and RPL involves comparisons between non nested models. In this study we focus on comparisons of non-nested models by means of standard model selection criteria (MSC) and three tests based on the Kuellback-Leibner Information Criterion (KLIC): the Vuong, the Horowitz and the Ben-Akiva and Swait’s tests. The results indicate that some penalization criteria work better than others; that model selection should be based on deterministic criteria complemented by tests; in particular, by the Vuong’s test, which has a low power, but it virtually never selects the wrong model