Mixup Without Hesitation
This incremental improvement addresses efficiency and ease-of-use issues in mixup for image classification and other tasks, benefiting practitioners in machine learning and computer vision.
The paper tackles the drawbacks of mixup, which requires more training epochs and hyperparameter tuning, by proposing mixup Without hesitation (mWh), a training algorithm that gradually replaces mixup with basic data augmentation to balance exploration and exploitation, achieving strong baselines with less training time and no hyperparameter search.
Mixup linearly interpolates pairs of examples to form new samples, which is easy to implement and has been shown to be effective in image classification tasks. However, there are two drawbacks in mixup: one is that more training epochs are needed to obtain a well-trained model; the other is that mixup requires tuning a hyper-parameter to gain appropriate capacity but that is a difficult task. In this paper, we find that mixup constantly explores the representation space, and inspired by the exploration-exploitation dilemma in reinforcement learning, we propose mixup Without hesitation (mWh), a concise, effective, and easy-to-use training algorithm. We show that mWh strikes a good balance between exploration and exploitation by gradually replacing mixup with basic data augmentation. It can achieve a strong baseline with less training time than original mixup and without searching for optimal hyper-parameter, i.e., mWh acts as mixup without hesitation. mWh can also transfer to CutMix, and gain consistent improvement on other machine learning and computer vision tasks such as object detection. Our code is open-source and available at https://github.com/yuhao318/mwh