Towards CGAN-based Satellite Image Synthesis with Partial Pixel-Wise Annotation
This addresses a challenge in satellite image synthesis for remote sensing applications, but it is incremental as it builds on existing CGAN methods.
The paper tackles the problem of generating high-quality satellite images using Conditional Generative Adversarial Nets (CGANs) when only partial pixel-wise annotations are available, proposing a detail augmentation solution that compensates for missing annotations by augmenting images with Canny edges or assigning colors to unannotated pixels.
Conditional Generative Adversarial Nets (CGANs) need a significantly huge dataset with a detailed pixel-wise annotation to generate high-quality images. Unfortunately, any amount of missing pixel annotations may significantly impact the result not only locally, but also in annotated areas. To the best of our knowledge, such a challenge has never been investigated in the broader field of GANs. In this paper, we take the first step in this direction to study the problem of CGAN-based satellite image synthesis given partially annotated images. We first define the problem of image synthesis using partially annotated data, and we discuss a scenario in which we face such a challenge. We then propose an effective solution called detail augmentation to address this problem. To do so, we tested two different approaches to augment details to compensate for missing pixel-wise annotations. In the first approach, we augmented the original images with their Canny edges to using the CGAN to compensate for the missing annotations. The second approach, however, attempted to assign a color to all pixels with missing annotation. Eventually, a different CGAN was trained to translate the new feature images into a final output.