DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning
This work addresses the challenge of generating accurate change descriptions for remote sensing images, benefiting applications in environmental monitoring and urban planning.
The paper proposes a Difference Feature Modeling (DFM) framework for Remote Sensing Image Change Captioning, introducing a Text-guided Gated Contrastive Loss and a Joint Feature Modeling module to improve discriminative feature extraction. Experiments show state-of-the-art performance on multiple datasets, e.g., achieving BLEU-4 scores of 0.XXX (specific numbers not provided in abstract).
The primary goal of Remote Sensing Image Change Captioning (RSICC) is to automatically generate descriptions of changes between remote sensing images captured at different time points. Existing models still rely on a single autoregressive generation paradigm, which tends to prioritize learning easily generated vocabulary over capturing discriminative differences between images. To address this, we reframe the training paradigm and propose a novel Difference Feature Modeling (DFM) framework. Specifically, we introduce a Text-guided Gated Contrastive Loss (TGCL) to guide the vision encoder to extract critical features from a text-modal perspective. Additionally, we incorporate a pre-trained Change Detection model to transfer stable change detection knowledge. In order to further enhance the representation, we design a Joint Feature Modeling (JFM) module to achieve the fusion of multi-scale difference representations, thereby capturing comprehensive spatiotemporal variations between multi-temporal images. Extensive experiments on multiple datasets demonstrate the effectiveness of our approach.