Image-Feature Weak-to-Strong Consistency: An Enhanced Paradigm for Semi-Supervised Learning
This work addresses the problem of enhancing semi-supervised learning performance for machine learning practitioners by introducing incremental improvements to the consistency paradigm.
The paper tackled the problem of improving semi-supervised learning by addressing limitations in image-level weak-to-strong consistency, such as excessive naive samples, and introduced an image-feature weak-to-strong consistency paradigm with feature-level perturbations and a triple-branch structure, resulting in significant performance enhancements for existing SSL algorithms on multiple benchmarks.
Image-level weak-to-strong consistency serves as the predominant paradigm in semi-supervised learning~(SSL) due to its simplicity and impressive performance. Nonetheless, this approach confines all perturbations to the image level and suffers from the excessive presence of naive samples, thus necessitating further improvement. In this paper, we introduce feature-level perturbation with varying intensities and forms to expand the augmentation space, establishing the image-feature weak-to-strong consistency paradigm. Furthermore, our paradigm develops a triple-branch structure, which facilitates interactions between both types of perturbations within one branch to boost their synergy. Additionally, we present a confidence-based identification strategy to distinguish between naive and challenging samples, thus introducing additional challenges exclusively for naive samples. Notably, our paradigm can seamlessly integrate with existing SSL methods. We apply the proposed paradigm to several representative algorithms and conduct experiments on multiple benchmarks, including both balanced and imbalanced distributions for labeled samples. The results demonstrate a significant enhancement in the performance of existing SSL algorithms.