AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentation
This work tackles the problem of improving semantic segmentation performance for rare classes in long-tail distributions, which is crucial for applications requiring accurate recognition of infrequent objects or regions.
This paper addresses the challenge of optimizing Area Under the ROC Curve (AUC) for pixel-level long-tail semantic segmentation, a task complicated by coupled loss terms and high memory demands for mini-batch estimation. The authors developed a pixel-level AUC loss function and a Tail-Classes Memory Bank (T-Memory Bank) to manage memory, demonstrating effectiveness across various benchmarks.
The Area Under the ROC Curve (AUC) is a well-known metric for evaluating instance-level long-tail learning problems. In the past two decades, many AUC optimization methods have been proposed to improve model performance under long-tail distributions. In this paper, we explore AUC optimization methods in the context of pixel-level long-tail semantic segmentation, a much more complicated scenario. This task introduces two major challenges for AUC optimization techniques. On one hand, AUC optimization in a pixel-level task involves complex coupling across loss terms, with structured inner-image and pairwise inter-image dependencies, complicating theoretical analysis. On the other hand, we find that mini-batch estimation of AUC loss in this case requires a larger batch size, resulting in an unaffordable space complexity. To address these issues, we develop a pixel-level AUC loss function and conduct a dependency-graph-based theoretical analysis of the algorithm's generalization ability. Additionally, we design a Tail-Classes Memory Bank (T-Memory Bank) to manage the significant memory demand. Finally, comprehensive experiments across various benchmarks confirm the effectiveness of our proposed AUCSeg method. The code is available at https://github.com/boyuh/AUCSeg.