Adapting SAM via Cross-Entropy Masking for Class Imbalance in Remote Sensing Change Detection
This addresses the problem of detecting changes in remote sensing images with imbalanced classes, which is incremental as it builds on existing foundational models.
The paper tackles class imbalance in remote sensing change detection by adapting the Segment Anything Model (SAM) with a novel cross-entropy masking loss, achieving a 2.5% F1-score improvement on the S2Looking dataset and outperforming state-of-the-art methods on four datasets.
Foundational models have achieved significant success in diverse domains of computer vision. They learn general representations that are easily transferable to tasks not seen during training. One such foundational model is Segment anything model (SAM), which can accurately segment objects in images. We propose adapting the SAM encoder via fine-tuning for remote sensing change detection (RSCD) along with spatial-temporal feature enhancement (STFE) and multi-scale decoder fusion (MSDF) to detect changes robustly at multiple scales. Additionally, we propose a novel cross-entropy masking (CEM) loss to handle high class imbalance in change detection datasets. Our method outperforms state-of-the-art (SOTA) methods on four change detection datasets, Levir-CD, WHU-CD, CLCD, and S2Looking. We achieved 2.5% F1-score improvement on a large complex S2Looking dataset. The code is available at: https://github.com/humza909/SAM-CEM-CD