Compensation Learning in Semantic Segmentation
This work addresses challenges in model development and data annotation for semantic segmentation, offering a method to handle label noise and class ambiguities, though it appears incremental as it builds on existing segmentation frameworks.
The paper tackles label noise and class ambiguities in semantic segmentation by proposing a compensation learning framework that adds a learned bias to classification logits and uses an uncertainty branch to apply it only to relevant regions, resulting in improved robustness against label noise and better identification of ambiguities across multiple datasets.
Label noise and ambiguities between similar classes are challenging problems in developing new models and annotating new data for semantic segmentation. In this paper, we propose Compensation Learning in Semantic Segmentation, a framework to identify and compensate ambiguities as well as label noise. More specifically, we add a ground truth depending and globally learned bias to the classification logits and introduce a novel uncertainty branch for neural networks to induce the compensation bias only to relevant regions. Our method is employed into state-of-the-art segmentation frameworks and several experiments demonstrate that our proposed compensation learns inter-class relations that allow global identification of challenging ambiguities as well as the exact localization of subsequent label noise. Additionally, it enlarges robustness against label noise during training and allows target-oriented manipulation during inference. We evaluate the proposed method on %the widely used datasets Cityscapes, KITTI-STEP, ADE20k, and COCO-stuff10k.