FourierSR: A Fourier Token-based Plugin for Efficient Image Super-Resolution
This work addresses the problem of limited receptive fields and high computational cost in image super-resolution for applications requiring efficient processing, though it is incremental as it builds on existing token mix technologies.
The paper tackles the challenge of improving super-resolution efficiency by proposing FourierSR, a Fourier token-based plugin that achieves an average PSNR gain of 0.34dB on the Manga109 test set at x4 scale, with only a 0.6% increase in parameters and 1.5% increase in FLOPs.
Image super-resolution (SR) aims to recover low-resolution images to high-resolution images, where improving SR efficiency is a high-profile challenge. However, commonly used units in SR, like convolutions and window-based Transformers, have limited receptive fields, making it challenging to apply them to improve SR under extremely limited computational cost. To address this issue, inspired by modeling convolution theorem through token mix, we propose a Fourier token-based plugin called FourierSR to improve SR uniformly, which avoids the instability or inefficiency of existing token mix technologies when applied as plug-ins. Furthermore, compared to convolutions and windows-based Transformers, our FourierSR only utilizes Fourier transform and multiplication operations, greatly reducing complexity while having global receptive fields. Experimental results show that our FourierSR as a plug-and-play unit brings an average PSNR gain of 0.34dB for existing efficient SR methods on Manga109 test set at the scale of x4, while the average increase in the number of Params and FLOPs is only 0.6% and 1.5% of original sizes. We will release our codes upon acceptance.