Exploring the Capability of Text-to-Image Diffusion Models with Structural Edge Guidance for Multi-Spectral Satellite Image Inpainting
This addresses satellite image processing for remote sensing applications, but it is incremental as it builds on existing models like StableDiffusion and ControlNet.
The paper tackled multi-spectral satellite image inpainting by exploring text-to-image diffusion models with structural edge guidance and RGB-to-MSI translation, but found that StableDiffusion-based inpainting produced artifacts and a self-supervised internal method achieved higher quality synthesis.
The letter investigates the utility of text-to-image inpainting models for satellite image data. Two technical challenges of injecting structural guiding signals into the generative process as well as translating the inpainted RGB pixels to a wider set of MSI bands are addressed by introducing a novel inpainting framework based on StableDiffusion and ControlNet as well as a novel method for RGB-to-MSI translation. The results on a wider set of data suggest that the inpainting synthesized via StableDiffusion suffers from undesired artifacts and that a simple alternative of self-supervised internal inpainting achieves a higher quality of synthesis.