94% on CIFAR-10 in 3.29 Seconds on a Single GPU
This work addresses the need for faster and cheaper machine learning research, particularly for researchers using CIFAR-10, though it is incremental as it builds on existing training methods.
The paper tackles the problem of reducing training time and cost for CIFAR-10 experiments by introducing methods that achieve 94% accuracy in 3.29 seconds, 95% in 10.4 seconds, and 96% in 46.3 seconds on a single GPU.
CIFAR-10 is among the most widely used datasets in machine learning, facilitating thousands of research projects per year. To accelerate research and reduce the cost of experiments, we introduce training methods for CIFAR-10 which reach 94% accuracy in 3.29 seconds, 95% in 10.4 seconds, and 96% in 46.3 seconds, when run on a single NVIDIA A100 GPU. As one factor contributing to these training speeds, we propose a derandomized variant of horizontal flipping augmentation, which we show improves over the standard method in every case where flipping is beneficial over no flipping at all. Our code is released at https://github.com/KellerJordan/cifar10-airbench.