Online Hyper-parameter Learning for Auto-Augmentation Strategy
This work addresses the problem of expensive data augmentation search for deep learning practitioners, offering a more economical solution, though it appears incremental as it builds on prior auto-augmentation methods.
The paper tackles the high computational cost of offline auto-augmentation strategies by proposing OHL-Auto-Aug, which learns augmentation policies online during network training, resulting in 60x faster search on CIFAR-10 and 24x faster on ImageNet while maintaining competitive accuracy.
Data augmentation is critical to the success of modern deep learning techniques. In this paper, we propose Online Hyper-parameter Learning for Auto-Augmentation (OHL-Auto-Aug), an economical solution that learns the augmentation policy distribution along with network training. Unlike previous methods on auto-augmentation that search augmentation strategies in an offline manner, our method formulates the augmentation policy as a parameterized probability distribution, thus allowing its parameters to be optimized jointly with network parameters. Our proposed OHL-Auto-Aug eliminates the need of re-training and dramatically reduces the cost of the overall search process, while establishes significantly accuracy improvements over baseline models. On both CIFAR-10 and ImageNet, our method achieves remarkable on search accuracy, 60x faster on CIFAR-10 and 24x faster on ImageNet, while maintaining competitive accuracies.