Northwest Evaluation Association

The applications of item response theory (IRT) models assume local item indepen-dence and that examinees are independent of each other. When a representative sample for psychometric analysis is selected using a cluster sampling method in a testlet-based assessment, both local item dependence and local person dependence are likely to be induced. This study proposed a four-level IRT model to simultane-ously account for dual local dependence due to item clustering and person cluster-ing. Model parameter estimation was explored using the Markov Chain Monte Carlo method. Model parameter recovery was evaluated in a simulation study in compar-ison with three other related models: the Rasch model, the Rasch testlet model, and the three-level Rasch model for person clustering. In general, the proposed model recovered the item difficulty and person ability parameters with the least total er-ror. The bias in both item and person parameter estimation was not affected but the standard error (SE) was affected. In some simulation conditions, the difference in classification accuracy between models could go up to 11%. The illustration using the real data generally supported model performance observed in the simulatio