Negative sampling in semi-supervised learning
This work addresses the challenge of enhancing semi-supervised learning efficiency and accuracy for machine learning practitioners, though it appears incremental as it builds upon existing methods like VAT and MixMatch.
The paper tackles the problem of improving semi-supervised learning by introducing Negative Sampling in Semi-Supervised Learning (NS3L), a simple and fast algorithm that, when added to state-of-the-art methods like VAT and MixMatch, leads to significant performance gains on benchmark datasets such as CIFAR10, CIFAR100, SVHN, and STL10.
We introduce Negative Sampling in Semi-Supervised Learning (NS3L), a simple, fast, easy to tune algorithm for semi-supervised learning (SSL). NS3L is motivated by the success of negative sampling/contrastive estimation. We demonstrate that adding the NS3L loss to state-of-the-art SSL algorithms, such as the Virtual Adversarial Training (VAT), significantly improves upon vanilla VAT and its variant, VAT with Entropy Minimization. By adding the NS3L loss to MixMatch, the current state-of-the-art approach on semi-supervised tasks, we observe significant improvements over vanilla MixMatch. We conduct extensive experiments on the CIFAR10, CIFAR100, SVHN and STL10 benchmark datasets.