Robinson's Measure of Agreement as a Parallel Forms Reliability Coefficient

A major deficiency in classical test theory is the reliance on Pearson product- moment (PPM) correlation concepts in the definition of reliability. PPA measures are totally insensitive to first moment differences in tests which leads to the dubious assumption of essential tan-equivalence. Robinson proposed a measure of agreement that is sensitive to different test difficulty and gives a practical statistic to estimate reliability in the presence of known form variation in difficulty. Robinson's measure of agreement appears to be a useful alternative to the generalizability coefficient, as it provides a more conservative estimate of reliability under conditions of parallel form differences in mean. This is likely to be especially useful when examining inter rater reliability when internal consistency of the raters is poor. Robinson's measure does not seem advantageous for highly reliable parallel tests such as are encountered in standardized testing programs. A simulation study is presented to illustrate the degree of the coefficient's sensitivity to form difficulty variance. Robinson,s measure of agreement and the intraciass correlation are coaputed for each simulation and their values are compared. (author/RL) Reproeluctions supplied by EDRS are the best that can be made from the original document