SwinFuSR: an image fusion-inspired model for RGB-guided thermal image super-resolution
This addresses the limitation of thermal imaging for applications like surveillance or medical imaging, though it is incremental as it builds on existing guided SR methods.
The paper tackles the problem of low-resolution thermal images by proposing SwinFuSR, a guided super-resolution model using Swin transformers, which achieves state-of-the-art performance in PSNR and SSIM metrics and placed 3rd in a 2024 challenge.
Thermal imaging plays a crucial role in various applications, but the inherent low resolution of commonly available infrared (IR) cameras limits its effectiveness. Conventional super-resolution (SR) methods often struggle with thermal images due to their lack of high-frequency details. Guided SR leverages information from a high-resolution image, typically in the visible spectrum, to enhance the reconstruction of a high-res IR image from the low-res input. Inspired by SwinFusion, we propose SwinFuSR, a guided SR architecture based on Swin transformers. In real world scenarios, however, the guiding modality (e.g. RBG image) may be missing, so we propose a training method that improves the robustness of the model in this case. Our method has few parameters and outperforms state of the art models in terms of Peak Signal to Noise Ratio (PSNR) and Structural SIMilarity (SSIM). In Track 2 of the PBVS 2024 Thermal Image Super-Resolution Challenge, it achieves 3rd place in the PSNR metric. Our code and pretained weights are available at https://github.com/VisionICLab/SwinFuSR.