ECMNet:Lightweight Semantic Segmentation with Efficient CNN-Mamba Network
This work addresses the problem of balancing accuracy and efficiency in semantic segmentation for vision applications, presenting an incremental improvement over existing methods.
The paper tackles semantic segmentation by proposing ECMNet, a lightweight Efficient CNN-Mamba Network that combines CNN and Mamba in a capsule-based framework to improve global context modeling, achieving 70.6% mIoU on Cityscapes and 73.6% mIoU on CamVid with 0.87M parameters and 8.27G FLOPs.
In the past decade, Convolutional Neural Networks (CNNs) and Transformers have achieved wide applicaiton in semantic segmentation tasks. Although CNNs with Transformer models greatly improve performance, the global context modeling remains inadequate. Recently, Mamba achieved great potential in vision tasks, showing its advantages in modeling long-range dependency. In this paper, we propose a lightweight Efficient CNN-Mamba Network for semantic segmentation, dubbed as ECMNet. ECMNet combines CNN with Mamba skillfully in a capsule-based framework to address their complementary weaknesses. Specifically, We design a Enhanced Dual-Attention Block (EDAB) for lightweight bottleneck. In order to improve the representations ability of feature, We devise a Multi-Scale Attention Unit (MSAU) to integrate multi-scale feature aggregation, spatial aggregation and channel aggregation. Moreover, a Mamba enhanced Feature Fusion Module (FFM) merges diverse level feature, significantly enhancing segmented accuracy. Extensive experiments on two representative datasets demonstrate that the proposed model excels in accuracy and efficiency balance, achieving 70.6% mIoU on Cityscapes and 73.6% mIoU on CamVid test datasets, with 0.87M parameters and 8.27G FLOPs on a single RTX 3090 GPU platform.