Transfer and Share: Semi-Supervised Learning from Long-Tailed Data
This addresses the challenge of efficient and effective semi-supervised learning from long-tailed data, which is common in real-world applications, by improving minority class performance without complex optimization or information loss.
The paper tackles the problem of learning from class-imbalanced data with few annotations in Long-Tailed Semi-Supervised Learning (LTSSL) by proposing TRAS, which transforms pseudo-label distributions to enhance minority class signals and shares feature extractors, resulting in much higher accuracy than state-of-the-art methods across all classes and minority classes.
Long-Tailed Semi-Supervised Learning (LTSSL) aims to learn from class-imbalanced data where only a few samples are annotated. Existing solutions typically require substantial cost to solve complex optimization problems, or class-balanced undersampling which can result in information loss. In this paper, we present the TRAS (TRAnsfer and Share) to effectively utilize long-tailed semi-supervised data. TRAS transforms the imbalanced pseudo-label distribution of a traditional SSL model via a delicate function to enhance the supervisory signals for minority classes. It then transfers the distribution to a target model such that the minority class will receive significant attention. Interestingly, TRAS shows that more balanced pseudo-label distribution can substantially benefit minority-class training, instead of seeking to generate accurate pseudo-labels as in previous works. To simplify the approach, TRAS merges the training of the traditional SSL model and the target model into a single procedure by sharing the feature extractor, where both classifiers help improve the representation learning. According to extensive experiments, TRAS delivers much higher accuracy than state-of-the-art methods in the entire set of classes as well as minority classes.