How to Construct More Accurate Student Models: Comparing and Optimizing Knowledge Tracing and Performance Factor Analysis

Abstract. Student modeling is a fundamental concept applicable to a variety of intelligent tutoring systems (ITS). However, there is not a lot of practical guidance on how to construct and train such models. This paper compares two approaches for student modeling, Knowledge Tracing (KT) and Performance Factors Analysis (PFA), at predicting individual student trials. We explore the space of design decisions for each approach and find a set of “best practices ” for each. In a head to head comparison, we find that PFA has considerably higher predictive accuracy than KT. In addition to being more accurate, we found that PFA’s parameter estimates were more plausible. Our best-performing model was a variant of PFA that ignored the tutor’s transfer model; that is, it assumed all skills influenced performance on all problems. Two possibilities are that this result is a general one suggesting we consider models that incorporate information from more than the typical handful of skills per problem, or that the transfer model our tutor uses is particularly weak