Region Rebalance for Long-Tailed Semantic Segmentation
This addresses the problem of long-tailed class distributions in semantic segmentation for computer vision applications, representing an incremental improvement.
The paper tackles class imbalance in semantic segmentation by proposing a region rebalance scheme, which groups pixel features into region features and applies a rebalanced classifier, achieving a +0.7% mIoU gain on the ADE20K val set with BEiT.
In this paper, we study the problem of class imbalance in semantic segmentation. We first investigate and identify the main challenges of addressing this issue through pixel rebalance. Then a simple and yet effective region rebalance scheme is derived based on our analysis. In our solution, pixel features belonging to the same class are grouped into region features, and a rebalanced region classifier is applied via an auxiliary region rebalance branch during training. To verify the flexibility and effectiveness of our method, we apply the region rebalance module into various semantic segmentation methods, such as Deeplabv3+, OCRNet, and Swin. Our strategy achieves consistent improvement on the challenging ADE20K and COCO-Stuff benchmark. In particular, with the proposed region rebalance scheme, state-of-the-art BEiT receives +0.7% gain in terms of mIoU on the ADE20K val set.