Overfitting of neural nets under class imbalance: Analysis and improvements for segmentation

Overfitting in deep learning has been the focus of a num-ber of recent works, yet its exact impact on the behaviour of neuralnetworks is not well understood. This study analyzes overfitting by ex-amining how the distribution of logits alters in relation to how muchthe model overfits. Specifically, we find that when training with few datasamples, the distribution of logit activations when processing unseen testsamples of an under-represented class tends to shift towards and evenacross the decision boundary, while the over-represented class seems un-affected. In image segmentation, foreground samples are often heavilyunder-represented. We observe that sensitivity of the model drops asa result of overfitting, while precision remains mostly stable. Based onour analysis, we derive asymmetric modifications of existing loss func-tions and regularizers including a large margin loss, focal loss, adver-sarial training and mixup, which specifically aim at reducing the shiftobserved when embedding unseen samples of the under-represented class.We study the case of binary segmentation of brain tumor core and showthat our proposed simple modifications lead to significantly improvedsegmentation performance over the symmetric variants