Journey Towards Tiny Perceptual Super-Resolution
This work addresses the deployment bottleneck of large perceptual super-resolution models for end devices, representing an incremental improvement in efficiency.
The paper tackles the problem of making perceptual super-resolution models more efficient for on-device deployment by proposing a neural architecture search approach that integrates NAS and GANs, resulting in tiny models that outperform SRGAN and EnhanceNet in perceptual and distortion metrics while being up to 26.4x more memory efficient and 33.6x more compute efficient.
Recent works in single-image perceptual super-resolution (SR) have demonstrated unprecedented performance in generating realistic textures by means of deep convolutional networks. However, these convolutional models are excessively large and expensive, hindering their effective deployment to end devices. In this work, we propose a neural architecture search (NAS) approach that integrates NAS and generative adversarial networks (GANs) with recent advances in perceptual SR and pushes the efficiency of small perceptual SR models to facilitate on-device execution. Specifically, we search over the architectures of both the generator and the discriminator sequentially, highlighting the unique challenges and key observations of searching for an SR-optimized discriminator and comparing them with existing discriminator architectures in the literature. Our tiny perceptual SR (TPSR) models outperform SRGAN and EnhanceNet on both full-reference perceptual metric (LPIPS) and distortion metric (PSNR) while being up to 26.4$\times$ more memory efficient and 33.6$\times$ more compute efficient respectively.