Comparison between DINA model and confirmatory noncompensatory MIRT model

Cognitive diagnostic models (CDM) are widely used to diagnose whether or not students master specific fine-grained skills. Multidimensional item response theory (MIRT) models adopt a continuous scale to locate students’ ability position based on their performance on test items. The current study aims to evaluate the cognitive diagnostic performance of the confirmatory noncompensatory MIRT (C-NMIRT) model by comparing it with the deterministic input, noisy ‘‘and’’ gate (DINA) model. A cutoff point is needed to transform continuous latent traits in the C-NMIRT model into categorical ones: mastery and no-mastery. A pilot study was conducted and 0 was found to be the proper cutoff point. Then, two simulation studies were conducted, where datasets were generated by the C-NMIRT model in the first study and generated by the DINA model in the second study. Both the DINA model and the C-NMIRT model were used for cognitive diagnosis and their results from both simulation studies were compared. The sample size N was 3000. Nine conditions were studied: three attribute numbers (K = 2, 3, 4) and three test lengths (short = 10, medium = 30, long = 50). Pattern correct classification rates (PCCRs) and the attribute correct classification rates (ACCRs) were calculated for estimation accuracy. Overall, estimation accuracy rates for both models increased as the attribute number decreased or test length enlarged. In addition, the first study found that estimation accuracy rates of two models were close in all of the nine conditions (discrepant rates were less than 3%). In addition, the second study indicated that the C-NMIRT model achieved similar estimation accuracy rates as the DINA model in the conditions with medium and long test lengths (discrepant rates were less than 3%). The C-NMIRT model also provided precisely estimated latent traits, which was beneficial for detailed skill mastery status analysis in educational settings