Model-building with multiply imputed data

Model selection is well-known for introducing additional uncertainty which can be more severe in the presence of missing data. Model averaging is an alternative to model selection which is intended to overcome the under-estimation of standard errors that is a consequence of model selection. Model selection and model averaging were explored on multiply-imputed data sets in terms of model selection and prediction. Three different model selection approaches (RR, STACK and M-STACK) and model averaging using three model-building strategies (non-overlapping variable sets, inclusive and restrictive strategies) to combine results from multiply-imputed data sets were explored using a basic Monte Carlo simulation study on linear and generalized linear models. The results showed that the STACK method performs better than RR and M-STACK in terms of model selection and prediction, whereas model averaging performs slightly better than STACK in terms of prediction. The inclusive and restrictive strategies perform better in terms of prediction but non-overlapping variable sets performs better for model selection. In conclusion, researchers should use STACK (with non-overlapping variable sets) for analysing data with missing values to determine which variables to include when making predictions but use model averaging (with a restrictive strategy) for prediction