An Augmented Reality Platform with Hand Gestures-based Navigation for Applications in Image-Guided Surgery: Prospective Concept Evaluation by Surgeons

Augmented reality (AR) technologies provide new scenarios for analysis and medical data representation, by combining the standard image-based analysis using 2D-slices with 3D virtual models of patient-specific structures of the body overlapped in the view of the real world in real-time. In this paper, we present the development of a generic AR-based and wireless gestures platform for image-guided surgery planning. The system was assessed with two minimally invasive surgery scenarios: heart mitral valve repair and endovascular brain aneurysm repair using CT Angiography (CTA) images. Twenty-one experiments with heart surgeons and neurosurgeons were carried out, testing the clinical pathology, anatomy exploration, interaction, and perceived usability. The tasks and scenarios, perceived illusion related to the realism of the 3D virtual models and hand manipulation, and the navigation experience were all well evaluated. Prospective applications were recognized as potentially useful and feasible for enhancing planning and trans-operative interventions for all the participants. Experiments performed showed that surgeons found the application a suitable alternative to understand better the previous steps involved in surgical planning, especially for minimally invasive surgery