The Role of Data Quality and Heterogeneity on the Calibration of Neural Networks

Neural networks have been widely studied and used in recent years due to its highclassification accuracy and training efficiency. With the increase of network depth, however,the models become worse calibrated, meaning they cannot reflect the true probabilities. Onthe other hand, in many applications such as medical diagnosis, facial recognition and selfdriving cars, the calibrated output probabilities are of critical importance. Therefore, theunderstanding of the cause of deep neural network uncalibration is of much concern.The influence of model structures on the output calibration has been explored.However, the impact of the training dataset quality and heterogeneity, such as dataset sizeand label noise remains unclear. In this thesis, the impact of data quality and heterogeneityon the output calibration is investigated theoretically and experimentally. Afterwards, thedefect of calibration methods using single global parameter are discussed. To overcomethe calibration issues resulting from the dataset heterogeneity, we propose an improvedcalibration technique that can give better performance