Exploring Sparsity in Image Super-Resolution for Efficient Inference
This work addresses efficiency issues in image super-resolution for applications on mobile devices, representing an incremental improvement over existing methods.
The paper tackles the problem of computational redundancy in CNN-based image super-resolution by introducing a Sparse Mask SR (SMSR) network that learns to skip unnecessary computations, achieving state-of-the-art performance with 41%, 33%, and 27% reductions in FLOPs for x2, x3, and x4 super-resolution respectively.
Current CNN-based super-resolution (SR) methods process all locations equally with computational resources being uniformly assigned in space. However, since missing details in low-resolution (LR) images mainly exist in regions of edges and textures, less computational resources are required for those flat regions. Therefore, existing CNN-based methods involve redundant computation in flat regions, which increases their computational cost and limits their applications on mobile devices. In this paper, we explore the sparsity in image SR to improve inference efficiency of SR networks. Specifically, we develop a Sparse Mask SR (SMSR) network to learn sparse masks to prune redundant computation. Within our SMSR, spatial masks learn to identify "important" regions while channel masks learn to mark redundant channels in those "unimportant" regions. Consequently, redundant computation can be accurately localized and skipped while maintaining comparable performance. It is demonstrated that our SMSR achieves state-of-the-art performance with 41%/33%/27% FLOPs being reduced for x2/3/4 SR. Code is available at: https://github.com/LongguangWang/SMSR.