Complex sampling design settings: Bridging technical documentation and applied researcher software use

This work documents our understanding of and best practice recommendations for specifying complex sampling design settings in the statistical software used for design-based survey data analyses. We discuss complex survey data features such as stratification, clustering, unequal probabilities of selection and calibration; outline their impact on estimation procedures; and demonstrate the dangers of incorrect analysis that ignores them. We show how statistical software handles these features. We analyze the documentation that accompanies several surveys collected and distributed by national statistical offices, statistical agencies and academic organizations, and rate the ease or difficulty of specifying the sampling design correctly in the software based on the dataset documentation provided. We conclude with our recommendations for survey data providers on how to create the transparent, accurate and effective documentation required by survey data users to account for sampling design features in their analyses of complex survey data. This work is co-authored with Brady T. West (University of Michigan) and Peter Lugtig (Utrecht University)