Imperfect Segmentation Labels: How Much Do They Matter?
This addresses the problem of imperfect segmentation labels, particularly in medical imaging, for researchers and practitioners, but it is incremental as it analyzes existing methods rather than introducing new ones.
The study investigated how different types and degrees of label errors in training data affect semantic segmentation model performance, finding that performance declines with boundary-localized errors, with U-Net being more robust to jagged boundary errors than SegNet and FCN32, while all architectures were robust to non-boundary-localized errors.
Labeled datasets for semantic segmentation are imperfect, especially in medical imaging where borders are often subtle or ill-defined. Little work has been done to analyze the effect that label errors have on the performance of segmentation methodologies. Here we present a large-scale study of model performance in the presence of varying types and degrees of error in training data. We trained U-Net, SegNet, and FCN32 several times for liver segmentation with 10 different modes of ground-truth perturbation. Our results show that for each architecture, performance steadily declines with boundary-localized errors, however, U-Net was significantly more robust to jagged boundary errors than the other architectures. We also found that each architecture was very robust to non-boundary-localized errors, suggesting that boundary-localized errors are fundamentally different and more challenging problem than random label errors in a classification setting.