Design and Analysis in a Cognitive Assessment Design and Analysis in a Cognitive Assessment

There is growing interest in educational assessments that coordinate substantive considerations, learning psychology, task design, and measurement models. This paper concerns an analysis of responses from an assessment of mixed-number subtraction that was created by Kikumi Tatsuoka (1983) in light of cognitive analyses of students ’ problem solutions. In particular, we fit a binary-skills multivariate latent class model to the data and compare results to those obtained with an item response theory model and a modified latent class model suggested by model criticism indices. Markov chain Monte Carlo (MCMC) techniques are used to estimate the parameters in the model in a Bayesian framework that integrates information from substantive theory, expert judgment, and empirical data. Key words: binary-skills model, cognitive diagnosis, latent class, Markov chain Monte Carlo, psychometrics i