What Are Effective Labels for Augmented Data? Improving Calibration and Robustness with AutoLabel
This addresses a specific issue in data augmentation for neural networks, offering an incremental improvement to existing techniques like RandAug, AugMix, and adversarial training.
The paper tackles the problem of labeling augmented data, showing that using one-hot labels for highly distorted data can degrade accuracy and calibration, and proposes AutoLabel to automatically learn label confidence based on transformation distance, which significantly improves calibration and accuracy for models under distributional shift on datasets like CIFAR-10, CIFAR-100, and ImageNet.
A wide breadth of research has devised data augmentation approaches that can improve both accuracy and generalization performance for neural networks. However, augmented data can end up being far from the clean training data and what is the appropriate label is less clear. Despite this, most existing work simply uses one-hot labels for augmented data. In this paper, we show re-using one-hot labels for highly distorted data might run the risk of adding noise and degrading accuracy and calibration. To mitigate this, we propose a generic method AutoLabel to automatically learn the confidence in the labels for augmented data, based on the transformation distance between the clean distribution and augmented distribution. AutoLabel is built on label smoothing and is guided by the calibration-performance over a hold-out validation set. We successfully apply AutoLabel to three different data augmentation techniques: the state-of-the-art RandAug, AugMix, and adversarial training. Experiments on CIFAR-10, CIFAR-100 and ImageNet show that AutoLabel significantly improves existing data augmentation techniques over models' calibration and accuracy, especially under distributional shift.