Test-time Cost-and-Quality Controllable Arbitrary-Scale Super-Resolution with Variable Fourier Components
This addresses the need for flexible super-resolution in applications where cost and quality trade-offs are critical, offering an incremental improvement over existing arbitrary-scale methods.
The paper tackles the problem of arbitrary-scale super-resolution with controllable computational cost and quality at test time, proposing an RNN-based method that adjusts Fourier components to achieve a lower PSNR drop compared to other state-of-the-art methods.
Super-resolution (SR) with arbitrary scale factor and cost-and-quality controllability at test time is essential for various applications. While several arbitrary-scale SR methods have been proposed, these methods require us to modify the model structure and retrain it to control the computational cost and SR quality. To address this limitation, we propose a novel SR method using a Recurrent Neural Network (RNN) with the Fourier representation. In our method, the RNN sequentially estimates Fourier components, each consisting of frequency and amplitude, and aggregates these components to reconstruct an SR image. Since the RNN can adjust the number of recurrences at test time, we can control the computational cost and SR quality in a single model: fewer recurrences (i.e., fewer Fourier components) lead to lower cost but lower quality, while more recurrences (i.e., more Fourier components) lead to better quality but more cost. Experimental results prove that more Fourier components improve the PSNR score. Furthermore, even with fewer Fourier components, our method achieves a lower PSNR drop than other state-of-the-art arbitrary-scale SR methods.