Computer-aided classification of colorectal segments during colonoscopy: a deep learning approach based on images of a magnetic endoscopic positioning device

Assessment of the anatomical colorectal segment of polyps during colonoscopy is important for treatment and follow-up strategies, but is largely operator dependent. This feasibility study aimed to assess whether, using images of a magnetic endoscope imaging (MEI) positioning device, a deep learning approach can be useful to objectively divide the colorectum into anatomical segments. Models based on the VGG-16 based convolutional neural network architecture were developed to classify the colorectum into anatomical segments. These models were pre-trained on ImageNet data and further trained using prospectively collected data of the POLAR study in which endoscopists were using MEI (3930 still images and 90,151 video frames). Five-fold cross validation with multiple runs was used to evaluate the overall diagnostic accuracies of the models for colorectal segment classification (divided into a 5-class and 2-class colorectal segment division). The colorectal segment assignment by endoscopists was used as the reference standard. For the 5-class colorectal segment division, the best performing model correctly classified the colorectal segment in 753 of the 1196 polyps, corresponding to an overall accuracy of 63%, sensitivity of 63%, specificity of 89% and kappa of 0.47. For the 2-class colorectal segment division, 1112 of the 1196 polyps were correctly classified, corresponding to an accuracy of 93%, sensitivity of 93%, specificity of 90% and kappa of 0.82. The diagnostic performance of a deep learning approach for colorectal segment classification based on images of a MEI device is yet suboptimal (clinicaltrials.gov: NCT03822390).