Designing a Hybrid Neural System to Learn Real-world Crack Segmentation from Fractal-based Simulation
This work addresses the challenge of limited and costly annotated data for crack segmentation in structural integrity assessment, representing an incremental improvement in domain-specific computer vision.
The paper tackled the problem of robust crack segmentation in concrete infrastructure by introducing a high-fidelity fractal-based simulator and a hybrid neural system that learns from simulation to handle real-world data, achieving effective segmentation as demonstrated empirically.
Identification of cracks is essential to assess the structural integrity of concrete infrastructure. However, robust crack segmentation remains a challenging task for computer vision systems due to the diverse appearance of concrete surfaces, variable lighting and weather conditions, and the overlapping of different defects. In particular recent data-driven methods struggle with the limited availability of data, the fine-grained and time-consuming nature of crack annotation, and face subsequent difficulty in generalizing to out-of-distribution samples. In this work, we move past these challenges in a two-fold way. We introduce a high-fidelity crack graphics simulator based on fractals and a corresponding fully-annotated crack dataset. We then complement the latter with a system that learns generalizable representations from simulation, by leveraging both a pointwise mutual information estimate along with adaptive instance normalization as inductive biases. Finally, we empirically highlight how different design choices are symbiotic in bridging the simulation to real gap, and ultimately demonstrate that our introduced system can effectively handle real-world crack segmentation.