Exploring the quality of skill-mastery prediction from Bayesian network learner models for smartphone-based paediatric care training in low-resource settings

There remains a lack of evidence on the implementation of Intelligent Tutoring Systems in low-resource settings, and in particular, of how adaptive instructional support can be implemented to support clinical training on smartphone devices. A core part of this challenge to determine an appropriate data and modelling approach to support adaptive instruction on mobile devices. Using data from a serious-gaming smartphone application for clinical training from Sub-Saharan Africa, this paper investigates models to support prediction of learner performance as a precursor to determining skill-mastery level. We explore Bayesian graph model configurations that predict unseen learner responses based on seen responses and investigate different combinations of these models that factor in time on task, and previous cumulative learning opportunities respectively. Our results show that a modelling approach that predicts learner performance while considering previous learning opportunities is more accurate than approaches that predicts learner performance based on time they spent on a learning task. Using time-on-task in combination with previous learning opportunities to augment prediction of learner performance produced no substantive increase on prediction accuracy compared to just using previous learning opportunities only. We discuss how our findings provide an avenue for introduction of adaptive scaffolding of feedback instruction, based on probabilistic performance thresholds informed by cumulative tries from previous attempts with the goal of helping the learner gain skill-mastery