MambaVF: State Space Model for Efficient Video Fusion
This addresses the computational bottleneck in video fusion for applications like multi-exposure and medical imaging, offering a more efficient alternative to existing methods.
The paper tackles the computational inefficiency of video fusion methods by introducing MambaVF, a state space model-based framework that eliminates optical flow estimation, achieving state-of-the-art performance across multiple benchmarks while reducing parameters by up to 92.25% and computational FLOPs by 88.79% with a 2.1x speedup.
Video fusion is a fundamental technique in various video processing tasks. However, existing video fusion methods heavily rely on optical flow estimation and feature warping, resulting in severe computational overhead and limited scalability. This paper presents MambaVF, an efficient video fusion framework based on state space models (SSMs) that performs temporal modeling without explicit motion estimation. First, by reformulating video fusion as a sequential state update process, MambaVF captures long-range temporal dependencies with linear complexity while significantly reducing computation and memory costs. Second, MambaVF proposes a lightweight SSM-based fusion module that replaces conventional flow-guided alignment via a spatio-temporal bidirectional scanning mechanism. This module enables efficient information aggregation across frames. Extensive experiments across multiple benchmarks demonstrate that our MambaVF achieves state-of-the-art performance in multi-exposure, multi-focus, infrared-visible, and medical video fusion tasks. We highlight that MambaVF enjoys high efficiency, reducing up to 92.25% of parameters and 88.79% of computational FLOPs and a 2.1x speedup compared to existing methods. Project page: https://mambavf.github.io