BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment
This addresses the problem of restoring high-quality images from noisy, misaligned low-resolution bursts for applications like photography, but it is incremental as it builds on existing methods like deformable convolutions and transformers.
The paper tackles the Burst Super-Resolution task by proposing BSRT, a new architecture that combines Pyramid Flow-Guided Deformable Convolution and Swin Transformer, achieving state-of-the-art results and winning the NTIRE2022 challenge.
This work addresses the Burst Super-Resolution (BurstSR) task using a new architecture, which requires restoring a high-quality image from a sequence of noisy, misaligned, and low-resolution RAW bursts. To overcome the challenges in BurstSR, we propose a Burst Super-Resolution Transformer (BSRT), which can significantly improve the capability of extracting inter-frame information and reconstruction. To achieve this goal, we propose a Pyramid Flow-Guided Deformable Convolution Network (Pyramid FG-DCN) and incorporate Swin Transformer Blocks and Groups as our main backbone. More specifically, we combine optical flows and deformable convolutions, hence our BSRT can handle misalignment and aggregate the potential texture information in multi-frames more efficiently. In addition, our Transformer-based structure can capture long-range dependency to further improve the performance. The evaluation on both synthetic and real-world tracks demonstrates that our approach achieves a new state-of-the-art in BurstSR task. Further, our BSRT wins the championship in the NTIRE2022 Burst Super-Resolution Challenge.