MarginMatch: Improving Semi-Supervised Learning with Pseudo-Margins
This addresses the challenge of low data regimes in semi-supervised learning for vision tasks, representing an incremental advance over existing methods.
The paper tackles the problem of improving semi-supervised learning by introducing MarginMatch, which uses training dynamics to measure pseudo-label quality, resulting in error rate improvements of 3.25% on CIFAR-100 with 25 labels per class and 3.78% on STL-10 with 4 labels per class.
We introduce MarginMatch, a new SSL approach combining consistency regularization and pseudo-labeling, with its main novelty arising from the use of unlabeled data training dynamics to measure pseudo-label quality. Instead of using only the model's confidence on an unlabeled example at an arbitrary iteration to decide if the example should be masked or not, MarginMatch also analyzes the behavior of the model on the pseudo-labeled examples as the training progresses, to ensure low quality predictions are masked out. MarginMatch brings substantial improvements on four vision benchmarks in low data regimes and on two large-scale datasets, emphasizing the importance of enforcing high-quality pseudo-labels. Notably, we obtain an improvement in error rate over the state-of-the-art of 3.25% on CIFAR-100 with only 25 labels per class and of 3.78% on STL-10 using as few as 4 labels per class. We make our code available at https://github.com/tsosea2/MarginMatch.