Burst Image Super-Resolution with Mamba
This work addresses the problem of enhancing image resolution from burst sequences for applications in photography and vision, offering a more efficient alternative to transformer-based methods with linear complexity.
The paper tackled burst image super-resolution by introducing BurstMamba, a Mamba-based architecture that decouples the task into spatial and temporal branches with novel strategies like optical flow-based serialization and wavelet-based reparameterization, achieving state-of-the-art performance on public benchmarks such as SyntheticSR, RealBSR-RGB, and RealBSR-RAW.
Burst image super-resolution (BISR) aims to enhance the resolution of a keyframe by leveraging information from multiple low-resolution images captured in quick succession. In the deep learning era, BISR methods have evolved from fully convolutional networks to transformer-based architectures, which, despite their effectiveness, suffer from the quadratic complexity of self-attention. We see Mamba as the next natural step in the evolution of this field, offering a comparable global receptive field and selective information routing with only linear time complexity. In this work, we introduce BurstMamba, a Mamba-based architecture for BISR. Our approach decouples the task into two specialized branches: a spatial module for keyframe super-resolution and a temporal module for subpixel prior extraction, striking a balance between computational efficiency and burst information integration. To further enhance burst processing with Mamba, we propose two novel strategies: (i) optical flow-based serialization, which aligns burst sequences only during state updates to preserve subpixel details, and (ii) a wavelet-based reparameterization of the state-space update rules, prioritizing high-frequency features for improved burst-to-keyframe information passing. Our framework achieves SOTA performance on public benchmarks of SyntheticSR, RealBSR-RGB, and RealBSR-RAW.