Learning Skill-Defining Latent Space in Video-based Analysis of Surgical Expertise – A Multi-Stream Fusion Approach

Abstract. In recent years, surgical simulation has emerged at the forefront of new technologies for improving the education and training of surgical residents. To ob-jectively evaluate the surgical skills of the trainees and reduce the training cost, an automated method for rating the performance of the operator is critical. How-ever, automated evaluation of surgical skills in a video-based system, e.g., the FLS trainer box, is still a challenging task, both due to the lack of reliable visual fea-tures and the lack of analysis tools that bridge the semantic gap between the low-level visual features and the high-level surgical skills. This study attempts to find a latent space for the visual features for supporting more meaningful analysis of surgical skills. The approach employs multi-modality fusion and Canonical Corre-lation Analysis as the key techniques. Experiments were designed to evaluate the proposed approach. The results suggest that this is a promising direction