Learning to Learn Parameterized Classification Networks for Scalable Input Images
This work addresses the need for efficient and flexible image classification in resource-constrained environments, representing an incremental improvement in adaptive inference methods.
The paper tackles the problem of convolutional neural networks lacking predictable recognition behavior across different input resolutions, which hinders deployment flexibility. The result is a method that uses meta learners to generate weights for various input scales, achieving consistently better accuracy than individually trained models on ImageNet.
Convolutional Neural Networks (CNNs) do not have a predictable recognition behavior with respect to the input resolution change. This prevents the feasibility of deployment on different input image resolutions for a specific model. To achieve efficient and flexible image classification at runtime, we employ meta learners to generate convolutional weights of main networks for various input scales and maintain privatized Batch Normalization layers per scale. For improved training performance, we further utilize knowledge distillation on the fly over model predictions based on different input resolutions. The learned meta network could dynamically parameterize main networks to act on input images of arbitrary size with consistently better accuracy compared to individually trained models. Extensive experiments on the ImageNet demonstrate that our method achieves an improved accuracy-efficiency trade-off during the adaptive inference process. By switching executable input resolutions, our method could satisfy the requirement of fast adaption in different resource-constrained environments. Code and models are available at https://github.com/d-li14/SAN.