Seeing Roads Through Words: A Language-Guided Framework for RGB-T Driving Scene Segmentation
Addresses the challenge of robust road scene segmentation under adverse conditions for autonomous driving, with a dynamic fusion strategy that outperforms static fusion methods.
CLARITY proposes a language-guided dynamic fusion framework for RGB-Thermal semantic segmentation that adapts to scene illumination conditions, achieving new state-of-the-art results on MFNet with 62.3% mIoU and 77.5% mAcc.
Robust semantic segmentation of road scenes under adverse illumination, lighting, and shadow conditions remain a core challenge for autonomous driving applications. RGB-Thermal fusion is a standard approach, yet existing methods apply static fusion strategies uniformly across all conditions, allowing modality-specific noise to propagate throughout the network. Hence, we propose CLARITY that dynamically adapts its fusion strategy to the detected scene condition. Guided by vision-language model (VLM) priors, the network learns to modulate each modality's contribution based on the illumination state while leveraging object embeddings for segmentation, rather than applying a fixed fusion policy. We further introduce two mechanisms - one which preserves valid dark-object semantics that prior noise-suppression methods incorrectly discard, and a hierarchical decoder that enforces structural consistency across scales to sharpen boundaries on thin objects. Experiments on the MFNet dataset demonstrate that CLARITY establishes a new state-of-the-art (SOTA), achieving 62.3% mIoU and 77.5% mAcc.