Efficient ResNets: Residual Network Design
This work addresses model size efficiency for storage-limited devices like IoT/edge devices, but it is incremental as it modifies existing ResNet designs.
The authors tackled the problem of designing a ResNet model for CIFAR-10 image classification with a constraint of under 5 million parameters, achieving a test accuracy of 96.04%, which is higher than ResNet18 with over 11 million parameters.
ResNets (or Residual Networks) are one of the most commonly used models for image classification tasks. In this project, we design and train a modified ResNet model for CIFAR-10 image classification. In particular, we aimed at maximizing the test accuracy on the CIFAR-10 benchmark while keeping the size of our ResNet model under the specified fixed budget of 5 million trainable parameters. Model size, typically measured as the number of trainable parameters, is important when models need to be stored on devices with limited storage capacity (e.g. IoT/edge devices). In this article, we present our residual network design which has less than 5 million parameters. We show that our ResNet achieves a test accuracy of 96.04% on CIFAR-10 which is much higher than ResNet18 (which has greater than 11 million trainable parameters) when equipped with a number of training strategies and suitable ResNet hyperparameters. Models and code are available at https://github.com/Nikunj-Gupta/Efficient_ResNets.