Rebalanced Region-Pixel Loss for Concrete Defects Segmentation

The deep convolution neural network (DCNN) using cross-entropy loss has already achieved high accuracy on the single defect classification task. However, for multi-defect segmentation, the imbalance of categories makes the segmentation of small targets (cracks) not meticulous enough. It also has to adjust the weight of categories manually. This research proposes a region-pixel loss function, which uses a rebalanced training method to balance category weights automatically and classify small categories more accurately. First, we use a wall-climbing robot to obtain the color and depth (RGB-D) information of the surface. Then, we adopt the visual based simultaneous localization and mapping (V-SLAM) method to select key-frame as the input of DCNN. Finally, reconstructing the 3D model of the environment with the output segmentation results. At the same time, to address the weak links of human-computer interaction in the post-processing of defect monitoring, a fast searching method from global 3D model to local details is proposed so that the staff can calmly obtain the local defects' detailed image. Experiments show that our loss function improves the intersection over union (IoU) of small flaws on the verification set. The designed global-to-local interaction method has been applied in the actual dam surface detection and provided an automated solution for monitoring large concrete surfaces