IamNN: Iterative and Adaptive Mobile Neural Network for Efficient Image Classification
This work addresses efficiency for image classification tasks, but it is incremental as it builds directly on existing ResNet architectures.
The authors tackled the problem of computational inefficiency in deep residual networks by designing a smaller network with parameter sharing and adaptive computation time, resulting in a model that adapts computational cost to input complexity.
Deep residual networks (ResNets) made a recent breakthrough in deep learning. The core idea of ResNets is to have shortcut connections between layers that allow the network to be much deeper while still being easy to optimize avoiding vanishing gradients. These shortcut connections have interesting side-effects that make ResNets behave differently from other typical network architectures. In this work we use these properties to design a network based on a ResNet but with parameter sharing and with adaptive computation time. The resulting network is much smaller than the original network and can adapt the computational cost to the complexity of the input image.