Implementation of Optimal Design for Item Calibration in Computerized Adaptive Testing (Cat)

72 p.Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 2006.Item calibration in small sample conditions is an important research and practical issue. This thesis aims to solve this issue by implementing a promising method, optimal design. Contrary to traditional random item selection, optimal design matches examinees and pretest items for maximal information on item parameters according to certain optimal design criterion. For example, in L-optimal design, item selection is based on correct item response probabilities equal to 25% or 75% with equal weights. A two-stage design is adopted for applying optimal design: random item selection as the first stage, and optimal design as the second stage with initial item parameter estimation obtained from the first stage. For controlling the measurement error in ability estimate which is an important issue when pretest items are seeded at the earlier stages of a CAT, optimal design uses two types of MLEs of ability levels: using current ability estimates in calculating item response probabilities at the item selection step, and using the final ability estimates at the item parameter estimation step. Random item selection uses the moment reconstruction (MR) method in logistic regression estimation. The results show that both of the methods of measurement error control are effective. Optimal design provides less biased item parameter estimation than estimation methods under random item selection (including MML estimation). The implications for item calibration, the research limitations, and future research directions are discussed. This exploratory study builds a solid basis for the future application of optimal design for accurate item calibration in small sample conditions in CATs.U of I OnlyRestricted to the U of I community idenfinitely during batch ingest of legacy ETD