Surgical data science in endoscopic surgery

Surgical data science (SDS) is a research field that aims to improve the quality of interventional healthcare by observing all aspects of the patient treatment process to provide the right assistance at the right time. To date, most SDS applications are based on the deep learning technique, which has shown great potential to solve challenging tasks in a complex surgical environment.However, such algorithms are dependent on a large amount of training data, which not only must contain data, but also labels (e.g., localization of an instrument in the image), so that they can be used for training. To date, however, such a mass of training data is not available. This is primarily because the creation of such data would often require medical experts, as well as significant time and money resources. This data scarcity motivates the two major challenges of surgical data science, namely, (1) how algorithms based on machine learning methods can be trained despite the limited availability of such data and (2) how more training data could be provided. For this work, as a concrete example of a surgical data science application, instrument segmentation of medical instruments in images of laparoscopic videos was used. This thesis investigated several means to alleviate this data scarcity in the context of laparoscopic instrument segmentation resulting in the following main contributions: First, it was examined how \textbf{unlabeled data can be integrated into the training of a machine-based algorithm to reduce the amount of annotated data}. Although with this method, the performance of a deep learning model trained on only a few labeled data could be significantly increased, the achieved performance was not high enough to cover the lack of training data. For this reason, as second contribution, \textbf{the largest dataset to date for the segmentation of multiple instruments in images of laparoscopic videos was created}. Generating the dataset followed a strict annotation protocol and was quality controlled. The created data was then \textbf{published as part of an international challenge to test the submitted methods and identify unresolved problems}. The third contribution was that \textbf{image characteristics were determined which negatively affect the segmentation quality}. In order to identify and quantify the influence of such characteristics, a statistical method has been developed. This analysis then flowed into the last contribution, \textbf{the targeted development of an algorithm that was designed to address the identified difficult characteristics} and achieved the best performance on the challenge dataset. As a result, this work provided a new tool for dealing with data sparsity by revealing the great potential of unlabeled data and the performance gain that can be achieved when generating high-quality datasets. Further, it showed that an in-depth statistical analysis of challenge results could be used to identify open issues of state-of-the-art methods and develop algorithms that are specifically designed to address those issues. This problem-driven approach even leads to a new best score on the task of multi-instance segmentation. Based on this thesis's results, one can confidently assume that the combination of generating data and problem-driven algorithm development and design has the potential to bridge the gap between research and the transition into clinical practice