Smocam : Smooth conditional attention mask for 3D-regression models

Despite the pervasive growth of deep neural networks in medical image analysis, methods to monitor and assess network outputs, such as segmentation or regression, remain limited. In this paper, we introduce SMOCAM (SMOoth Conditional Attention Mask), an optimization method that reveals the specific regions of the input image taken into account by the prediction of a trained neural network. We developed SMOCAM explicitly to perform saliency analysis for complex regression tasks in 3D medical imagery. Our formulation optimises an 3D-attention mask at a given layer of a convolutional neural network (CNN). Unlike previous attempts, our method is relatively fast (40s per output) and is suitable for large data such as 3D MRI. We applied SMOCAM on a CNN that predicts Brain morphometry from 3D MRI which was trained using more than 5000 3D brain MRIs. We show that SMOCAM highlights neural network's limitations when cases are underrepresented and in cases with large volume asymmetry.