Attention-based Dual Supervised Decoder for RGBD Semantic Segmentation
This work addresses the challenge of comprehensive multimodal fusion in RGBD segmentation, offering an incremental improvement for computer vision applications.
The paper tackles the problem of RGBD semantic segmentation by proposing an attention-based dual supervised decoder to better utilize multimodal information, achieving superior performance on NYUDv2 and SUN-RGBD datasets compared to state-of-the-art methods.
Encoder-decoder models have been widely used in RGBD semantic segmentation, and most of them are designed via a two-stream network. In general, jointly reasoning the color and geometric information from RGBD is beneficial for semantic segmentation. However, most existing approaches fail to comprehensively utilize multimodal information in both the encoder and decoder. In this paper, we propose a novel attention-based dual supervised decoder for RGBD semantic segmentation. In the encoder, we design a simple yet effective attention-based multimodal fusion module to extract and fuse deeply multi-level paired complementary information. To learn more robust deep representations and rich multi-modal information, we introduce a dual-branch decoder to effectively leverage the correlations and complementary cues of different tasks. Extensive experiments on NYUDv2 and SUN-RGBD datasets demonstrate that our method achieves superior performance against the state-of-the-art methods.