No Data Augmentation? Alternative Regularizations for Effective Training on Small Datasets
This work addresses the problem of limited data in computer vision for researchers and practitioners, offering an incremental improvement over existing methods.
The paper tackles the challenge of image classification on small datasets by exploring alternative regularization strategies, achieving 66.5% test accuracy on ciFAIR-10 with only 1% of the CIFAR-10 training data, matching state-of-the-art methods.
Solving image classification tasks given small training datasets remains an open challenge for modern computer vision. Aggressive data augmentation and generative models are among the most straightforward approaches to overcoming the lack of data. However, the first fails to be agnostic to varying image domains, while the latter requires additional compute and careful design. In this work, we study alternative regularization strategies to push the limits of supervised learning on small image classification datasets. In particular, along with the model size and training schedule scaling, we employ a heuristic to select (semi) optimal learning rate and weight decay couples via the norm of model parameters. By training on only 1% of the original CIFAR-10 training set (i.e., 50 images per class) and testing on ciFAIR-10, a variant of the original CIFAR without duplicated images, we reach a test accuracy of 66.5%, on par with the best state-of-the-art methods.