Exploiting Inherent Class Label: Towards Robust Scribble Supervised Semantic Segmentation
This work addresses the problem of reducing human annotation labor for semantic segmentation, but it is incremental as it builds on existing scribble-supervised methods with new modules and benchmarks.
The paper tackles the challenges of inconsistent predictions and annotation variability in scribble-based weakly supervised semantic segmentation by proposing a class-driven scribble promotion network, which achieves competitive performance in accuracy and robustness on new benchmarks like ScribbleCOCO and ScribbleCityscapes.
Scribble-based weakly supervised semantic segmentation leverages only a few annotated pixels as labels to train a segmentation model, presenting significant potential for reducing the human labor involved in the annotation process. This approach faces two primary challenges: first, the sparsity of scribble annotations can lead to inconsistent predictions due to limited supervision; second, the variability in scribble annotations, reflecting differing human annotator preferences, can prevent the model from consistently capturing the discriminative regions of objects, potentially leading to unstable predictions. To address these issues, we propose a holistic framework, the class-driven scribble promotion network, for robust scribble-supervised semantic segmentation. This framework not only utilizes the provided scribble annotations but also leverages their associated class labels to generate reliable pseudo-labels. Within the network, we introduce a localization rectification module to mitigate noisy labels and a distance perception module to identify reliable regions surrounding scribble annotations and pseudo-labels. In addition, we introduce new large-scale benchmarks, ScribbleCOCO and ScribbleCityscapes, accompanied by a scribble simulation algorithm that enables evaluation across varying scribble styles. Our method demonstrates competitive performance in both accuracy and robustness, underscoring its superiority over existing approaches. The datasets and the codes will be made publicly available.