BurstM: Deep Burst Multi-scale SR using Fourier Space with Optical Flow
This work addresses a specific bottleneck in MFSR for image processing applications, offering incremental improvements over existing methods.
The paper tackles misalignment and high-frequency representation issues in multi-frame super-resolution (MFSR) by proposing BurstM, which uses optical flow for alignment and Fourier coefficients for textures, achieving state-of-the-art performance with support for various scale factors.
Multi frame super-resolution(MFSR) achieves higher performance than single image super-resolution (SISR), because MFSR leverages abundant information from multiple frames. Recent MFSR approaches adapt the deformable convolution network (DCN) to align the frames. However, the existing MFSR suffers from misalignments between the reference and source frames due to the limitations of DCN, such as small receptive fields and the predefined number of kernels. From these problems, existing MFSR approaches struggle to represent high-frequency information. To this end, we propose Deep Burst Multi-scale SR using Fourier Space with Optical Flow (BurstM). The proposed method estimates the optical flow offset for accurate alignment and predicts the continuous Fourier coefficient of each frame for representing high-frequency textures. In addition, we have enhanced the network flexibility by supporting various super-resolution (SR) scale factors with the unimodel. We demonstrate that our method has the highest performance and flexibility than the existing MFSR methods. Our source code is available at https://github.com/Egkang-Luis/burstm