Item position effects and differential item functioning for African-American and White examinees completing the arithmetic reasoning subtest of the preliminary item tryout version of Form E of the General Aptitude Test Battery

Schmitt and Dorans (1990) hypothesized that one possible reason for apparent test bias on the Scholastic Aptitude Test (SAT) against African Americans was related to a difference in processing speed between White and African American examinees. They based this hypothesis on an analysis of data generated by the administration of the SAT. Specifically, if one omitted those items that African Americans did not reach, the differential item functioning that led to bias against African Americans virtually disappeared. This study utilized data collected during the preliminary item tryout phase of Form E of the General Aptitude Test Battery (GATB) to examine the extent to which item position effects might be linked to differential item functioning (DIF). During the item pretest study, a set of arithmetic reasoning items was presented to a nationwide sample of examinees. Roughly half of the examinees were presented the items in forward order and the other half were presented the items in reverse order. This allowed an evaluation of the extent to which an item\u27s ordinal position affected examinee performance. The study failed to provide direct evidence for Schmitts\u27 and Dorans\u27 (1990) hypothesis due to the fact that very little DIF was identified. However, several troubling observations emerged. First, White examinees answered 32 out of the 33 items correctly at a greater proportion than African American examinees (p $\u3c$.05). Second, there were statistically significant differences between how White and African American examinees selected incorrect item alternatives. Third, when evaluated using a biserial or point-biserial correlation coefficient as an estimator of the effectiveness of an item at discriminating between high and low ability examinees, the test items tended to predict scores for White examinees better than for African American examinees indicating that they may not be a valid measure of the African American examinees\u27 arithmetic reasoning ability. Finally, there is a significant correlation between item position and the difference between White and African American examinee performance. Items administered early in the examination tended to work more equally well for both groups while items administered later tended to work better for White examinees than for African American examinees