Balancing Logit Variation for Long-tailed Semantic Segmentation
This addresses the challenge of imbalanced class representation in semantic segmentation, which is a common issue in computer vision, though it appears incremental as it builds on existing approaches.
The paper tackles the problem of long-tailed data distribution in semantic segmentation by introducing category-wise variation into network predictions during training to balance feature distributions, resulting in improved performance across various datasets and state-of-the-art methods.
Semantic segmentation usually suffers from a long-tail data distribution. Due to the imbalanced number of samples across categories, the features of those tail classes may get squeezed into a narrow area in the feature space. Towards a balanced feature distribution, we introduce category-wise variation into the network predictions in the training phase such that an instance is no longer projected to a feature point, but a small region instead. Such a perturbation is highly dependent on the category scale, which appears as assigning smaller variation to head classes and larger variation to tail classes. In this way, we manage to close the gap between the feature areas of different categories, resulting in a more balanced representation. It is noteworthy that the introduced variation is discarded at the inference stage to facilitate a confident prediction. Although with an embarrassingly simple implementation, our method manifests itself in strong generalizability to various datasets and task settings. Extensive experiments suggest that our plug-in design lends itself well to a range of state-of-the-art approaches and boosts the performance on top of them.