Combined Image Data Augmentations diminish the benefits of Adaptive Label Smoothing
This work addresses the problem of optimizing regularization for image classifiers, but it is incremental as it builds on existing adaptive label smoothing methods.
The paper investigated extending adaptive label smoothing to aggressive augmentations like random erasing and noise injection, finding it permits stronger regularization but loses benefits with diverse transformations like TrivialAugment and harms robustness to corruptions.
Soft augmentation regularizes the supervised learning process of image classifiers by reducing label confidence of a training sample based on the magnitude of random-crop augmentation applied to it. This paper extends this adaptive label smoothing framework to other types of aggressive augmentations beyond random-crop. Specifically, we demonstrate the effectiveness of the method for random erasing and noise injection data augmentation. Adaptive label smoothing permits stronger regularization via higher-intensity Random Erasing. However, its benefits vanish when applied with a diverse range of image transformations as in the state-of-the-art TrivialAugment method, and excessive label smoothing harms robustness to common corruptions. Our findings suggest that adaptive label smoothing should only be applied when the training data distribution is dominated by a limited, homogeneous set of image transformation types.