Adaptive Importance Learning for Improving Lightweight Image Super-resolution Network
This work addresses the challenge of applying super-resolution to devices with limited computing power by optimizing training for lightweight networks, though it appears incremental as it focuses on a novel learning strategy rather than a new architecture.
The authors tackled the problem of improving the performance of lightweight image super-resolution networks by proposing an adaptive importance learning scheme that trains networks with an easy-to-complex paradigm, dynamically updating pixel importance based on training loss, which enhanced pixel-wise fitting capacity without specifying concrete performance numbers.
Deep neural networks have achieved remarkable success in single image super-resolution (SISR). The computing and memory requirements of these methods have hindered their application to broad classes of real devices with limited computing power, however. One approach to this problem has been lightweight network architectures that bal- ance the super-resolution performance and the computation burden. In this study, we revisit this problem from an orthog- onal view, and propose a novel learning strategy to maxi- mize the pixel-wise fitting capacity of a given lightweight network architecture. Considering that the initial capacity of the lightweight network is very limited, we present an adaptive importance learning scheme for SISR that trains the network with an easy-to-complex paradigm by dynam- ically updating the importance of image pixels on the basis of the training loss. Specifically, we formulate the network training and the importance learning into a joint optimization problem. With a carefully designed importance penalty function, the importance of individual pixels can be gradu- ally increased through solving a convex optimization problem. The training process thus begins with pixels that are easy to reconstruct, and gradually proceeds to more complex pixels as fitting improves.