SAR-to-RGB Translation with Latent Diffusion for Earth Observation
This addresses a practical issue for remote sensing applications where optical images are unavailable, but it is incremental as it applies existing diffusion methods to a specific domain.
The paper tackles the problem of missing optical (RGB) images in Earth observation due to cloud cover by proposing a diffusion model-based approach for SAR-to-RGB translation, generating synthetic optical images from SAR inputs. Results show that class conditioning improves land cover classification accuracy, and the method performs competitively in cloud removal tasks, though generated images may not perfectly replicate real data.
Earth observation satellites like Sentinel-1 (S1) and Sentinel-2 (S2) provide complementary remote sensing (RS) data, but S2 images are often unavailable due to cloud cover or data gaps. To address this, we propose a diffusion model (DM)-based approach for SAR-to-RGB translation, generating synthetic optical images from SAR inputs. We explore three different setups: two using Standard Diffusion, which reconstruct S2 images by adding and removing noise (one without and one with class conditioning), and one using Cold Diffusion, which blends S2 with S1 before removing the SAR signal. We evaluate the generated images in downstream tasks, including land cover classification and cloud removal. While generated images may not perfectly replicate real S2 data, they still provide valuable information. Our results show that class conditioning improves classification accuracy, while cloud removal performance remains competitive despite our approach not being optimized for it. Interestingly, despite exhibiting lower perceptual quality, the Cold Diffusion setup performs well in land cover classification, suggesting that traditional quantitative evaluation metrics may not fully reflect the practical utility of generated images. Our findings highlight the potential of DMs for SAR-to-RGB translation in RS applications where RGB images are missing.