Augmented reality : showing the way in orthopaedic surgery

Introduction and Objectives: Computer Assisted Orthopaedic Surgery (CAOS) techniques are increasingly being applied to the challenge of ensuring accurate implant positioning during Unicondylar Knee Arthroplasty (UKA). Both commercial systems in this field use bulky infrared stereoscopic tracking systems, that restrict patient access, and standard monitors that divide the surgeon’s attention away from the patient. We present a comparison of two prototype CAOS systems which utilise significantly smaller tracking systems and provide surgical guidance through Augmented Reality (AR). AR mixes both real and virtual imagery to offer a non-intrusive and intuitive form of guidance. Methods: Both systems are based upon six degrees of freedom tracking of fiducial markers using visible spectrum Microsoft Lifecam Studio web cameras, developed under the OpenCV library. The live video used for tracking is augmented in real-time with virtual elements, such as CT models and alignment guides, by a modern OpenGL engine. Passive control is provided by coloured resection models while additional active control is achieved through speed control of the cutting burr. The systems differ in their tracking and display systems. The first system uses a tool-mounted monoscopic camera with the augmented video displayed on a standard monitor. The second system uses a head-mounted stereoscopic camera and a head mounted display (HMD). The accuracy of both systems was determined by probing 108 known points on a 110x110x50 mm geometry. System latency was also measured, as the time delay between video capture and display of the final augmented frame. Finally the practicality, comfort and intuitiveness of guidance were gauged through user feedback. Results: The monoscopic and stereoscopic systems produced an accuracy of 1.5±0.3 and 2.1±0.6 mm, respectively. However, the validity of these results is questionable as it was found the test points were inaccurate, particularly in the z axis. The monoscopic system presented an average latency of 263±55 ms compared to 293±15 ms for the stereoscopic system. Both systems rated well for intuitiveness, with the natural view and 3D display of the stereoscopic system making some instructions clearer. The monoscopic system offered superior comfort. However, tasks such as hip centre location, which required two hands and tracking of multiple markers, were more practical with the stereoscopic system. Conclusion: Due to the inaccuracies of the current target a new 3D printed target is being produced with sub-50 μm accuracy. Based on the results currently available, however, both systems produced accuracies outwith the desired submillimetre target. It is felt that this should be reached with several basic modifications. With the advent of USB3.0 cameras, higher resolution images may be streamed at high frame rates. This will allow more accurate marker detection and should therefore improve pose estimation. Additionally, all experiments were performed using small 20 mm markers. Larger markers would provide more data points for detection and a wider point base for pose estimation, both of which offer a potential accuracy boost. Both systems produced latencies which were not readily perceivable to the user. This is particularly important for the HMD of the stereoscopic system, where large latencies may induce ‘simulator sickness’. It is felt that the hands-free and natural view offered by the stereoscopic system makes it the superior technology, particularly for use in a surgical environment. This is despite the inherent reduced accuracy resulting from tool to target tracking requiring an additional marker compared to the monoscopic system. If the desired accuracy is achieved it is hoped that the intuitiveness afforded by the augmented guidance will make the system appealing to healthcare professionals by keeping learning curves to a minimum. This will therefore increase the adoption of CAOS technologies