Quantity beats quality for semantic segmentation of corrosion in images
This work addresses the resource-intensive challenge of dataset creation for researchers developing deep learning models for detecting and locating specialist features like corrosion, though it is incremental as it compares existing methods on new data.
The study tackled the problem of dataset creation for semantic segmentation of corrosion in images, showing that a large, noisy dataset of 250 images labeled by undergraduates outperformed a small, expertly segmented dataset of 10 images labeled by subject matter experts, with results measured using mean Intersection over Union and micro F-score metrics after 50,000 epochs of training.
Dataset creation is typically one of the first steps when applying Artificial Intelligence methods to a new task; and the real world performance of models hinges on the quality and quantity of data available. Producing an image dataset for semantic segmentation is resource intensive, particularly for specialist subjects where class segmentation is not able to be effectively farmed out. The benefit of producing a large, but poorly labelled, dataset versus a small, expertly segmented dataset for semantic segmentation is an open question. Here we show that a large, noisy dataset outperforms a small, expertly segmented dataset for training a Fully Convolutional Network model for semantic segmentation of corrosion in images. A large dataset of 250 images with segmentations labelled by undergraduates and a second dataset of just 10 images, with segmentations labelled by subject matter experts were produced. The mean Intersection over Union and micro F-score metrics were compared after training for 50,000 epochs. This work is illustrative for researchers setting out to develop deep learning models for detection and location of specialist features.