Dual-domain Modulation Network for Lightweight Image Super-Resolution
This work addresses the need for efficient super-resolution models for applications with limited computational resources, offering a novel approach to frequency-based methods.
The paper tackles the problem of lightweight image super-resolution by proposing a Dual-domain Modulation Network that integrates wavelet and Fourier information to balance structure and high-frequency reconstruction while reducing computational costs. It achieves comparable PSNR to state-of-the-art methods with less than 50-60% of their FLOPs and inference speeds 15.4x and 5.4x faster.
Lightweight image super-resolution (SR) aims to reconstruct high-resolution images from low-resolution images under limited computational costs. We find that existing frequency-based SR methods cannot balance the reconstruction of overall structures and high-frequency parts. Meanwhile, these methods are inefficient for handling frequency features and unsuitable for lightweight SR. In this paper, we show that introducing both wavelet and Fourier information allows our model to consider both high-frequency features and overall SR structure reconstruction while reducing costs. Specifically, we propose a Dual-domain Modulation Network that integrates both wavelet and Fourier information for enhanced frequency modeling. Unlike existing methods that rely on a single frequency representation, our design combines wavelet-domain modulation via a Wavelet-domain Modulation Transformer (WMT) with global Fourier supervision, enabling complementary spectral learning well-suited for lightweight SR. Experimental results show that our method achieves a comparable PSNR to SRFormer and MambaIR while with less than 50\% and 60\% of their FLOPs and achieving inference speeds 15.4x and 5.4x faster, respectively, demonstrating the effectiveness of our method on SR quality and lightweight. Code link: https://github.com/24wenjie-li/DMNet