Semi-Supervised Learning by Label Gradient Alignment
It addresses the problem of limited labeled data for machine learning practitioners, but appears incremental as it builds on existing standardized architectures.
The paper tackles semi-supervised learning by introducing label gradient alignment, a method that imputes labels for unlabeled data and trains on them, achieving state-of-the-art accuracy on CIFAR-10 classification.
We present label gradient alignment, a novel algorithm for semi-supervised learning which imputes labels for the unlabeled data and trains on the imputed labels. We define a semantically meaningful distance metric on the input space by mapping a point (x, y) to the gradient of the model at (x, y). We then formulate an optimization problem whose objective is to minimize the distance between the labeled and the unlabeled data in this space, and we solve it by gradient descent on the imputed labels. We evaluate label gradient alignment using the standardized architecture introduced by Oliver et al. (2018) and demonstrate state-of-the-art accuracy in semi-supervised CIFAR-10 classification.