Detecting Cloud Presence in Satellite Images Using the RGB-based CLIP Vision-Language Model
This addresses cloud detection for satellite image processing, but it is incremental as it adapts an existing model to a new domain.
This work tackled cloud detection in satellite images by applying the pre-trained CLIP vision-language model, achieving non-trivial performance with generalization across datasets and sensors, and a low-cost fine-tuning stage significantly increased true negative rates.
This work explores capabilities of the pre-trained CLIP vision-language model to identify satellite images affected by clouds. Several approaches to using the model to perform cloud presence detection are proposed and evaluated, including a purely zero-shot operation with text prompts and several fine-tuning approaches. Furthermore, the transferability of the methods across different datasets and sensor types (Sentinel-2 and Landsat-8) is tested. The results that CLIP can achieve non-trivial performance on the cloud presence detection task with apparent capability to generalise across sensing modalities and sensing bands. It is also found that a low-cost fine-tuning stage leads to a strong increase in true negative rate. The results demonstrate that the representations learned by the CLIP model can be useful for satellite image processing tasks involving clouds.