An interactive instance segmentation system with multi‐resolution convolutional neural networks

Abstract In this paper, a fast interactive instance segmentation (IIS) system is proposed and it is composed of an effective heatmap generator, a multi‐resolution network (MRNet), and an adaptive threshold refiner to promptly and precisely predict the masks of the objects. The proposed heatmap generator after interaction clicks can help the MRNet to successfully learn the sensitive features for better prediction. Based on convolutional neural network models, the proposed MRNet backbone produces multiple features across multiple resolutions and can intrinsically predict the sharp contour of the object. After the probabilistic prediction achieved by the MRNet, the Otsu's threshold refiner is proposed to further remove some uncertain pixels in the predicted mask. Experimental results demonstrate that the proposed IIS system can promptly predict sharp masks of the targeted objects with mIoU of 89.1% in PASCAL VOC 2012 [1] validation set. Compared to other existing interactive methods, the proposed system can effectively predict the segmentation mask with higher accuracy and less interaction efforts