Item Discrimination and Type I Error in the Detection of Differential Item Functioning

In 2009, DeMars stated that when impact exists there will be Type I error inflation, especially with larger sample sizes and larger discrimination parameters for items. One purpose of this study is to present the patterns of Type I error rates using Mantel–Haenszel (MH) and logistic regression (LR) procedures when the mean ability between the focal and reference groups varies from zero to one standard deviation. The findings can be used as guides for alpha adjustment when using MH or LR meth-ods when impact exists. A second purpose is to better understand the conditions that cause Type I error rates to inflate. The results indicate that inflation can be con-trolled even in the presence of large ability differences and with large samples. Keywords differential item functioning, item discrimination, logistic regression, Mantel–Haenszel, Type I error The investigation of differential item functioning (DIF) has become an area of much interest in applied testing and methodological studies. DIF is said to exist when examinees with identical proficiency in two groups, traditionally labeled as reference group and focal group, perform differently on a given item after controlling for, or conditioning on, a measure of overall skill measured by the item (Dorans & Holland, 1993; Narayanan & Swaminathan, 1996). In many DIF analyses, the total test score is used as the measure of overall skill, and item performance is simply the proportion of persons having the item correct