Correcting effect sizes for score reliability: A reminder that measurement and substantive issues are linked inextricably

The present article emphasizes that measurement issues must be explicitly considered even in studies that focus on substantive questions. First, dynamics associated with insuf-ficient attention being paid to score reliabilities in substantive studies are discussed. Next, reasons to adjust effect size indices for score unreliability are presented. Finally, some procedures for adjusting effect sizes for score reliability are briefly reviewed. Most social science researchers are aware of the roughly 80-year-old controversy (cf. Boring, 1919) regarding statistical significance testing (Pedahazur & Schmelkin, 1991). Although there are thoughtful supporters of null hypothesis testing (e.g., Abelson, 1997; Cortina & Dunlap, 1997; Frick, 1996; Robinson & Wainer, in press), researchers increasingly recognize the limitations of the statistical method Rozeboom (1997) described as “the most bone-headed misguided procedure ever institutionalized in the rote training of science students ” (p. 335). This awakening is attributable to the efforts o