ARM: A Confidence-Based Adversarial Reweighting Module for Coarse Semantic Segmentation
This addresses the challenge of leveraging coarse annotations for semantic segmentation, offering a method to enhance model performance with noisy data, though it is incremental as it builds on existing reweighting strategies.
The paper tackles the problem of noisy coarse annotations in semantic segmentation by introducing a confidence-based adversarial reweighting module (ARM) that simultaneously mines valuable pixels and suppresses mislabeled pixels, improving mIoU on Cityscapes and achieving 47.50 mIoU on ADE20K.
Coarsely-labeled semantic segmentation annotations are easy to obtain, but therefore bear the risk of losing edge details and introducing background pixels. Impeded by the inherent noise, existing coarse annotations are only taken as a bonus for model pre-training. In this paper, we try to exploit their potentials with a confidence-based reweighting strategy. To expand, loss-based reweighting strategies usually take the high loss value to identify two completely different types of pixels, namely, valuable pixels in noise-free annotations and mislabeled pixels in noisy annotations. This makes it impossible to perform two tasks of mining valuable pixels and suppressing mislabeled pixels at the same time. However, with the help of the prediction confidence, we successfully solve this dilemma and simultaneously perform two subtasks with a single reweighting strategy. Furthermore, we generalize this strategy into an Adversarial Reweighting Module (ARM) and prove its convergence strictly. Experiments on standard datasets shows our ARM can bring consistent improvements for both coarse annotations and fine annotations. Specifically, built on top of DeepLabv3+, ARM improves the mIoU on the coarsely-labeled Cityscapes by a considerable margin and increases the mIoU on the ADE20K dataset to 47.50.