Real-time photo-realistic rendering for surgical simulations with graphics hardware

Abstract: Computer-based surgical simulations are being increasingly used for training and skills assessment. They provide an efficient and cost effective alternative to traditional training methods. To allow for both basic and advanced skills assessment, the required perceptual fidelity is essential to capturing the natural behavior of the operator. The level of realism in terms of object and scene appearance determines the faithfulness and hence the degree of immersion experienced by the trainee in the virtual world. This paper presents a novel photo-realistic rendering approach based on real-time per-pixel effects by using the graphics hardware. Improved realism is achieved by a combined use of specular reflectance and refractance maps to model the effect of surface details and mucous layer on the overall visual appearance of the tissue. The key steps involved in the proposed technique are described, and quantitative performance assessment results demonstrate the practical advantages of the proposed technique.