Auto-Prompting SAM for Weakly Supervised Landslide Extraction
This work addresses landslide mapping for remote sensing applications, offering a novel approach to improve segmentation accuracy with weak labels, though it is incremental in leveraging SAM.
The paper tackled the problem of imprecise boundaries in weakly supervised landslide extraction from remote sensing data by proposing APSAM, a method that auto-prompts the Segment Anything Model (SAM) using adaptive prompts from class activation maps, resulting in improvements of at least 3.0% in F1 score and 3.69% in IoU compared to state-of-the-art methods.
Weakly supervised landslide extraction aims to identify landslide regions from remote sensing data using models trained with weak labels, particularly image-level labels. However, it is often challenged by the imprecise boundaries of the extracted objects due to the lack of pixel-wise supervision and the properties of landslide objects. To tackle these issues, we propose a simple yet effective method by auto-prompting the Segment Anything Model (SAM), i.e., APSAM. Instead of depending on high-quality class activation maps (CAMs) for pseudo-labeling or fine-tuning SAM, our method directly yields fine-grained segmentation masks from SAM inference through prompt engineering. Specifically, it adaptively generates hybrid prompts from the CAMs obtained by an object localization network. To provide sufficient information for SAM prompting, an adaptive prompt generation (APG) algorithm is designed to fully leverage the visual patterns of CAMs, enabling the efficient generation of pseudo-masks for landslide extraction. These informative prompts are able to identify the extent of landslide areas (box prompts) and denote the centers of landslide objects (point prompts), guiding SAM in landslide segmentation. Experimental results on high-resolution aerial and satellite datasets demonstrate the effectiveness of our method, achieving improvements of at least 3.0\% in F1 score and 3.69\% in IoU compared to other state-of-the-art methods. The source codes and datasets will be available at https://github.com/zxk688.