Technical Skills Assessment in Robotic Surgery: Using Patient Outcomes to Set the Standard

Introduction: Outcome-based assessments form a central component of the new wave of educational reform in surgical residency. Evaluation of technical performance in the operating room is at the heart of the move to competency-based surgical education. Despite the numerous tools available to educators to utilize within a program of assessment, key questions remain unanswered on the application of these assessment scores, particularly around setting standards for high-stakes evaluations, and their impact on clinically meaningful outcomes. This thesis focuses on the use of technical skills assessment tools in robotic surgery, the available methods of standard setting in procedural skill, and the proposal of a novel benchmarking method that uses patients' outcomes to set the standard. Methods: Data were prospectively collected and analyzed using descriptive and inferential statistics (both parametric and non-parametric). Research designs contained within the thesis include: two systematic reviews in accordance with PRISMA guidelines, a case-control study, two methodological manuscripts, and a prospective multicenter cohort study. Contemporary validity frameworks were used throughout to provide validity evidence for all assessment tools. Results: Six studies comprise the thesis. Two systematic reviews collated, summarized, and appraised the existing literature on robotic-assisted surgery technical skills assessment strategies, and the methods used to create absolute standards in procedural technical skill. The case-control pilot study examined a single surgeon’s experience with robotic-assisted radical prostatectomy and provided important proof of the predictive relationship between technical skill and patient outcomes. A methodology paper utilized this predictive model to describe a method of standard setting that uses the clinical outcome as the fulcrum around which the technical performance benchmark is set. A prospective multicentre study used a diverse cohort of patients and surgeons to build predictive models using multiple assessment strategies across a number of clinical outcomes. Finally, an additional methodological study took the multiple predictive models to create a final, weighted composite score that can be used to set performance standards that control for important clinicopathological factors. Conclusion: This thesis describes the predictive relationship between technical performance in robotic surgery and patient clinical outcomes, and leverages this association to create a novel method of benchmarking surgical skill.Ph.D