The Effects of Item Grouping on Test Reliability

Optimal ordering of items on a multidimensional test has been the focus of several studies. In all but one study, previous research centered on measures of personality or opinion. The current study examines item grouping effects for a cognitive ability test. Two forms of a cognitive ability test containing four constructs (verbal ability, basic computation, number series, and spatial visualization) were prepared. Form A consisted of items grouped by construct, and Form B had items dispersed randomly throughout the test. The order of items within a construct remained the same for both forms. Tests were administered to 186 undergraduate psychology students. Coefficient alpha estimates of reliability on Form A were compared to coefficient alphas for the same constructs on Form B. Additionally, differences in mean scores across all four constructs and construct intercorrelations were compared by format. There were no significant differences in coefficient alphas and only one (basic computation) was in the hypothesized direction. There was only one significant construct intercorrelation pairing (basic computation vs. spatial visualization), and there were no significant differences in mean scores. Based on the lack of consistent findings, we found little support for a grouping effect for cognitive ability tests