Assessing the Impact of Characteristics of the Test, Common-items, and Examinees on the Preservation of Equity Properties in Mixed-format Test Equating

Preservation of equity properties was examined using four equating methods - IRT True Score, IRT Observed Score, Frequency Estimation, and Chained Equipercentile - in a mixed-format test under a common-item nonequivalent groups (CINEG) design. Equating of mixed-format tests under a CINEG design can be influenced by factors such as attributes of the test, the common-item set, and examinees. Additionally, unidimensionality may not hold due to the inclusion of multiple item formats. Different item formats could measure different latent constructs and thus cause a multidimensional test structure. The purpose of this study was to examine the impact of test structures (unidimensional versus within-item multidimensional as modeled through a bifactor model), differences in group ability distributions (equivalent versus nonequivalent), and characteristics of the common-item set (format representative versus non-representative) on each equating method’s ability to preserve equity properties. \ud The major findings can be summarized as follows: IRT equating methods outperformed traditional equating methods in terms of equity preservation across all conditions. Traditional equating methods performed similarly when groups were equivalent. However, large discrepancies between the methods were found as a direct function of an increase in mean group ability differences. The IRT true score method was most successful in terms of First-Order Equity preservation regardless of test structure. All methods preserved Second-Order Equity similarly under unidimensional test structures. The IRT true score method was superior to all other equating methods in terms of Second-Order Equity when the test structures were multidimensional. Similar results in terms of the Same Distribution property were obtained for each method when the groups were equivalent. The IRT observed score method was the best preserving when mean group ability differences increased. This was observed regardless of underlying test structure. Lower equity indices were observed when the common-item set was representative of the total test in particular when group differences were large. Similar patterns in terms of the performance of equating methods were observed regardless of the underlying test structure. \ud These results are discussed within the literature framework as it pertains to mixed-format test equating. Limitations of the current study are discussed and suggestions for future research are provided.\u