Multiple Defects Inspection of Dam Spillway Surface Using Deep Learning and 3D Reconstruction Techniques

After a lengthy period of scouring, the reinforced concrete surface of the dam spillway (i.e., drift spillways and flood discharge spillways) will suffer from deterioration and damage. Regular manual inspection is time-consuming and dangerous. This paper presents a robotic solution to detect automatically, count defect instance numbers, and reconstruct the surface of dam spillways by incorporating the deep learning method with a visual 3D reconstruction method. The lack of a real dam defect dataset and incomplete registration of minor defects on the 3D mesh model in fusion step are two challenges addressed in the paper. We created a multi-class semantic segmentation dataset of 1711 images (with resolutions of 848 × 480 and 1280 × 720 pixels) acquired by a wall-climbing robot, including cracks, erosion, spots, patched areas, and power safety cable. Then, the architecture of the U-net is modified with pixel-adaptive convolution (PAC) and conditional random field (CRF) to segment different scales of defects, trained, validated, and tested using this dataset. The reconstruction and recovery of minor defect instances in the flow surface and sidewall are facilitated using a keyframe back-projection method. By generating an instance adjacency matrix within the class, the intersection over union (IoU) of 3D voxels is calculated to fuse multiple instances. Our segmentation model achieves an average IoU of 60% for five defect class. For the surface model’s semantic recovery and instance statistics, our method achieves accurate statistics of patched area and erosion instances in an environment of 200 m², and the average absolute error of the number of spots and cracks has reduced from the original 13.5 to 3.5.

This creative work has been published under the Creative Commons Attribution 4.0 International (CC-BY 4.0) license which allows copying, and redistribution as well as adaptation of the original work provided appropriate credit is given to the original author and the conditions of the license are met.