Epsilon Consistent Mixup: Structural Regularization with an Adaptive Consistency-Interpolation Tradeoff
This work addresses semi-supervised learning challenges for researchers and practitioners, offering an incremental improvement over existing Mixup methods.
The paper tackles the problem of improving semi-supervised classification accuracy by proposing Epsilon Consistent Mixup (εmu), a data-based structural regularization technique that adaptively combines Mixup's linear interpolation with consistency regularization. The result shows improved accuracy on SVHN and CIFAR10 datasets, with the largest gains in low label-availability scenarios, though no specific numerical improvements are provided.
In this paper we propose $ε$-Consistent Mixup ($ε$mu). $ε$mu is a data-based structural regularization technique that combines Mixup's linear interpolation with consistency regularization in the Mixup direction, by compelling a simple adaptive tradeoff between the two. This learnable combination of consistency and interpolation induces a more flexible structure on the evolution of the response across the feature space and is shown to improve semi-supervised classification accuracy on the SVHN and CIFAR10 benchmark datasets, yielding the largest gains in the most challenging low label-availability scenarios. Empirical studies comparing $ε$mu and Mixup are presented and provide insight into the mechanisms behind $ε$mu's effectiveness. In particular, $ε$mu is found to produce more accurate synthetic labels and more confident predictions than Mixup.