Computerized Adaptive Testing---New Developments and Applications

92 p.Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 2008.Computerized adaptive testing (CAT) has become a very important testing mode since it was introduced into the testing field in the early 1970's. It has clear advantages over the traditional paper-pencil testing in many aspects including shorter tests and more efficient score reporting. However, it also raises new issues and challenges. This dissertation is an attempt to address some of those issues, which include: (1) Test security. When CAT is offered on a continuous basis, test security becomes a big concern. Large-scale on-line item theft has caused the termination of the CAT Graduate Record Exam (GRE) in several Asian countries and areas. Consequently, protecting test security becomes a crucial component of every CAT program that offers continuous testing. Chapter 2 introduces a probabilistic item selection algorithm based on the maximum entropy criterion to combat large-scale item theft. (2) Content-balancing and other non-statistical constraints. It is very important to ensure that every examinee receives a content-valid test, meaning that the test should be adequate and balanced in terms of content coverage. Meanwhile, other constraints such as item type balancing may also apply. Therefore, an item selection algorithm capable of managing multiple constraints becomes necessary. Chapter 3 proposes a new item selection algorithm, namely the maximum priority index (MPI) method, to address this problem. (3) Classification accuracy and consistency. The effect of item selection algorithms on the classification decisions made, such as passing/failure, is rarely examined in the CAT literature. It is rarely examined in the CAT literature. Chapter 4 discusses the challenges in obtaining classification accuracy and consistency indices in the context of CAT and gives an example to illustrate the computation of these two indices. (4) CAT for cognitive diagnosis. Chapter 5 proposes several new item-selection algorithms for cognitive diagnostic CAT and compares them with two methods developed by Xu et al. (2005). The new algorithms feature Bayesian item selection on the basis of likelihood-weighted Kullback-Leibler information and distance-weighted Kullback-Leibler information.U of I OnlyRestricted to the U of I community idenfinitely during batch ingest of legacy ETD