Not every day is a sunny day: Synthetic cloud injection for deep land cover segmentation robustness evaluation across data sources
This work addresses the challenge of reliable land cover segmentation in tropical regions with frequent clouds, offering incremental improvements for satellite data analysis.
The paper tackled the problem of land cover segmentation in satellite imagery being limited by cloud cover and detail loss in deep networks, by developing a cloud injection algorithm for robustness evaluation and a method to inject Normalized Difference Indices into decoding layers, resulting in performance improvements of up to 2.78% on cloud-free data and significant gains with radar-optical fusion under cloud-covered conditions.
Supervised deep learning for land cover semantic segmentation (LCS) relies on labeled satellite data. However, most existing Sentinel-2 datasets are cloud-free, which limits their usefulness in tropical regions where clouds are common. To properly evaluate the extent of this problem, we developed a cloud injection algorithm that simulates realistic cloud cover, allowing us to test how Sentinel-1 radar data can fill in the gaps caused by cloud-obstructed optical imagery. We also tackle the issue of losing spatial and/or spectral details during encoder downsampling in deep networks. To mitigate this loss, we propose a lightweight method that injects Normalized Difference Indices (NDIs) into the final decoding layers, enabling the model to retain key spatial features with minimal additional computation. Injecting NDIs enhanced land cover segmentation performance on the DFC2020 dataset, yielding improvements of 1.99% for U-Net and 2.78% for DeepLabV3 on cloud-free imagery. Under cloud-covered conditions, incorporating Sentinel-1 data led to significant performance gains across all models compared to using optical data alone, highlighting the effectiveness of radar-optical fusion in challenging atmospheric scenarios.