RS3Mamba: Visual State Space Model for Remote Sensing Images Semantic Segmentation
This work addresses semantic segmentation for remote sensing applications in geoscience, presenting an incremental improvement by adapting the novel Mamba architecture to this domain-specific task.
The authors tackled semantic segmentation of remote sensing images by proposing RS3Mamba, a dual-branch network that combines a convolution-based main branch with a visual state space (VSS) auxiliary branch to address limitations in long-range modeling and computational complexity. Experimental results on ISPRS Vaihingen and LoveDA Urban datasets demonstrate its effectiveness, achieving competitive performance with improved efficiency.
Semantic segmentation of remote sensing images is a fundamental task in geoscience research. However, there are some significant shortcomings for the widely used convolutional neural networks (CNNs) and Transformers. The former is limited by its insufficient long-range modeling capabilities, while the latter is hampered by its computational complexity. Recently, a novel visual state space (VSS) model represented by Mamba has emerged, capable of modeling long-range relationships with linear computability. In this work, we propose a novel dual-branch network named remote sensing images semantic segmentation Mamba (RS3Mamba) to incorporate this innovative technology into remote sensing tasks. Specifically, RS3Mamba utilizes VSS blocks to construct an auxiliary branch, providing additional global information to convolution-based main branch. Moreover, considering the distinct characteristics of the two branches, we introduce a collaborative completion module (CCM) to enhance and fuse features from the dual-encoder. Experimental results on two widely used datasets, ISPRS Vaihingen and LoveDA Urban, demonstrate the effectiveness and potential of the proposed RS3Mamba. To the best of our knowledge, this is the first vision Mamba specifically designed for remote sensing images semantic segmentation. The source code will be made available at https://github.com/sstary/SSRS.