Outcomes, measurement instruments and their validity evidence in randomised trials on virtual, augmented and mixed reality in undergraduate medical education: a systematic mapping review

Background: Extended reality, encompassing virtual, augmented and mixed reality, is increasingly used in medical education. Studies assessing effectiveness of these new educational modalities should measure relevant outcomes using outcome measurement tools with validity evidence. Our aim was to determine the choice of outcomes, measurement instruments and the use of measurement instruments with validity evidence in randomized controlled trials (RCTs) on the effectiveness of virtual reality (VR), augmented reality (AR) and mixed reality (MR) in medical student education. Methods: We conducted a systematic mapping review. We searched seven major bibliographic databases from January 1990 to April 2020. Two reviewers screened the citations and extracted data independently from the included studies. We report our findings in line with PRISMA guidelines. Results: Of 127 retrieved RCTs, 116 were on VR, eleven on AR. No RCTs on MR in medical student education were found. Of the studies on VR, 64 studies were on VR simulators, 31 studies on screen-based VR, 9 on VR patient simulations and 12 on VR serious games. Most studies reported only one outcome and immediate, post-intervention assessment data. Skills outcome was the most common outcome in studies on VR simulators, VR patient simulations and AR. Knowledge was the most common outcome in studies on screen-based VR and VR serious games. Less common outcomes included participants’ attitudes, satisfaction, cognitive or mental load, learning efficacy, engagement and/or self-efficacy beliefs, emotional state, competency developed and patient outcomes. At least one form of validity evidence was found in around half of the studies on VR simulator, VR patient simulations, VR serious games and AR, and in only a quarter of studies on screen-based VR. Most studies used assessment methods that were implemented in a non-digital format such as paper-based written exercises or in-person assessments where examiners observed performance. Conclusions: RCTs on VR and AR in medical education report a restricted range of outcomes, mostly skills and knowledge. Studies largely report immediate, post-intervention outcome data and use assessment methods that are in a non-digital format. Future RCTs should include broader set of outcomes, report on validity evidence employed measurement instruments and explore the use of assessments that are implemented digitally