Assessment of Uncertainty of Neural Net Predictions in the Tasks of Classification, Detection and Segmentation

This work focuses on comparing three widely used methods for improving uncertainty estimations: Deep Ensembles, Monte Carlo Dropout, and Temperature Scaling. These methods are applied to six computer vision models that are pretrained as well as trained from scratch. The models are then evaluated on computer vision datasets for classification, semantic segmentation, and object detection using a wide range of metrics. The models are also evaluated on distorted versions of these datasets to measure their performance on out-of-distribution data. These modified models achieve promising results. Ensembles outperform the other models by as high as 70 % in accuracy and 0.2 in IOU on the distorted MedSeg COVID-19 segmentation dataset while also outperforming the other models on the CIFAR-100 and FMNIST datasets