SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning
This work addresses a challenging and realistic scenario in semi-supervised learning for researchers and practitioners dealing with imbalanced and mismatched data distributions, representing a novel method rather than an incremental improvement.
The paper tackles the problem of realistic long-tailed semi-supervised learning, where labeled data is imbalanced and unlabeled data distribution is unknown and potentially mismatched, by proposing SimPro, a probabilistic framework that decouples conditional and marginal class distributions to estimate class distributions without predefined assumptions, achieving state-of-the-art performance across diverse benchmarks.
Recent advancements in semi-supervised learning have focused on a more realistic yet challenging task: addressing imbalances in labeled data while the class distribution of unlabeled data remains both unknown and potentially mismatched. Current approaches in this sphere often presuppose rigid assumptions regarding the class distribution of unlabeled data, thereby limiting the adaptability of models to only certain distribution ranges. In this study, we propose a novel approach, introducing a highly adaptable framework, designated as SimPro, which does not rely on any predefined assumptions about the distribution of unlabeled data. Our framework, grounded in a probabilistic model, innovatively refines the expectation-maximization (EM) algorithm by explicitly decoupling the modeling of conditional and marginal class distributions. This separation facilitates a closed-form solution for class distribution estimation during the maximization phase, leading to the formulation of a Bayes classifier. The Bayes classifier, in turn, enhances the quality of pseudo-labels in the expectation phase. Remarkably, the SimPro framework not only comes with theoretical guarantees but also is straightforward to implement. Moreover, we introduce two novel class distributions broadening the scope of the evaluation. Our method showcases consistent state-of-the-art performance across diverse benchmarks and data distribution scenarios. Our code is available at https://github.com/LeapLabTHU/SimPro.