Using Item Response Theory in Survey Research: Accommodating Complex Sampling Designs for Respondents and Items

Researchers in the social sciences, particularly in the field of attitudinal and health-outcomes survey research, have taken note of the relatively new opportunities and advancements modern item response theory (IRT) (e.g., Embretson & Reise, 2000) presents. A number of issues commonly encountered by survey methodologists prevent the direct adoption of basic IRT methods in contexts for which they were not originally designed, however. Consequently, despite its potential advantages, the current use of IRT methods in survey research is not widespread (Reeve & Mâsse, 2004). To facilitate a wider awareness, use, and development of IRT methods in survey research, this presentation explores necessary adaptations of traditional IRT methodologies to overcome implementation challenges in that field. It considers particularly the core statistical issues associated with complex sampling designs for respondents and items – unequal sampling probabilities, clustering, data missing by design, and context effects – with an eye towards developing computer-adaptive surveys measuring attitudes and perceptions. It demonstrates how the union of established theory and cutting-edge techniques in educational measurement and survey methodology can help researchers in social science fields take ful