Spectral Synthesis for Satellite-to-Satellite Translation
This addresses the challenge of building downstream applications in remote sensing by synchronizing multispectral data, though it is incremental as it builds on existing translation methods.
The paper tackles the problem of inconsistent spectral imagery across Earth-observing satellites by generating synthetic spectral bands as an unsupervised image-to-image translation task, showing that cross-domain reconstruction outperforms measurements from a second vantage point and improves cloud detection segmentation performance beyond a baseline.
Earth observing satellites carrying multi-spectral sensors are widely used to monitor the physical and biological states of the atmosphere, land, and oceans. These satellites have different vantage points above the earth and different spectral imaging bands resulting in inconsistent imagery from one to another. This presents challenges in building downstream applications. What if we could generate synthetic bands for existing satellites from the union of all domains? We tackle the problem of generating synthetic spectral imagery for multispectral sensors as an unsupervised image-to-image translation problem with partial labels and introduce a novel shared spectral reconstruction loss. Simulated experiments performed by dropping one or more spectral bands show that cross-domain reconstruction outperforms measurements obtained from a second vantage point. On a downstream cloud detection task, we show that generating synthetic bands with our model improves segmentation performance beyond our baseline. Our proposed approach enables synchronization of multispectral data and provides a basis for more homogeneous remote sensing datasets.