Supervised Deep Learning with Finite Element Synthetic Data for Force Estimation in Robotic-assisted Surgery

The prevalence of robot-assisted minimally invasive surgery on the liver has increased exponentially. Having accurate, real-time knowledge of force during robotic-assisted surgical procedures is vital for safe surgery. Many techniques have been proposed in the literature to tackle this concern, from deploying force sensors to physics-based modeling of the robot and, more recently, learning-based force prediction. For a high-fidelity force measurement, sensors should be integrated at the instrument's tip, close to the surgical site, which brings sterilization, biocompatibility, and MRI compatibility concerns. On the other hand, Dynamic robot modeling may be precise in a specific setting, but it suffers from the lack of generalization encountering unseen settings. Considering the drawbacks and deficits of mentioned methods, indirect force estimation via deflection measurement through imaging techniques is investigated as an alternative solution, generally done via machine learning methods. Almost all previous studies are either supervised learning, where data are labeled with ex-vivo ground truth, or unsupervised or semi-supervised learning, where outcomes are promising but not adequate. This study investigated indirect force prediction for the human liver through a developed deep autoencoder model as a supervised deep learning method trained via synthetic data generated by finite element (FE) simulation. This method took advantage of various patient-specific livers parameters and geometries extracted from CT images. The Hyperelastic modeling of the soft tissue is considered and assessed with various hyperelastic models. The uncertainty due to the surgical tool's occlusion is addressed in this model, and a novel state vector was proposed to improve the accuracy and generalisability of the prediction. In addition, the impact of the bounded region on the model's accuracy was evaluated. It was shown that the proposed method could predict the external force on an unseen tissue with different geometry and mechanical properties. The accuracy of force prediction considering tool occlusion noise diminishes by 4.2 percent, which is in an acceptable range. The accuracy of presented model for various scenarios ranges from 95 to 88 percent. Model's results have been evaluated by predicting the force encountering the surface deformation of an unseen liver geometry and constitutive model where the mean absolute error of prediction is 0.249 Newton