Hypernetwork-Based Augmentation
This work addresses the problem of slow data augmentation policy search for deep learning practitioners, offering an incremental improvement over AutoAugment.
The paper tackles the computational intensity of AutoAugment by proposing Hypernetwork-Based Augmentation (HBA), an efficient gradient-based search algorithm that learns model parameters and augmentation hyperparameters simultaneously, achieving competitive state-of-the-art results in search speed and accuracy on datasets like CIFAR-10, CIFAR-100, SVHN, and ImageNet.
Data augmentation is an effective technique to improve the generalization of deep neural networks. Recently, AutoAugment proposed a well-designed search space and a search algorithm that automatically finds augmentation policies in a data-driven manner. However, AutoAugment is computationally intensive. In this paper, we propose an efficient gradient-based search algorithm, called Hypernetwork-Based Augmentation (HBA), which simultaneously learns model parameters and augmentation hyperparameters in a single training. Our HBA uses a hypernetwork to approximate a population-based training algorithm, which enables us to tune augmentation hyperparameters by gradient descent. Besides, we introduce a weight sharing strategy that simplifies our hypernetwork architecture and speeds up our search algorithm. We conduct experiments on CIFAR-10, CIFAR-100, SVHN, and ImageNet. Our results show that HBA is competitive to the state-of-the-art methods in terms of both search speed and accuracy.