Convolutional LSTMs for Cloud-Robust Segmentation of Remote Sensing Imagery
This addresses the challenge of cloud interference in optical Earth observation for remote sensing applications, but it is incremental as it builds on prior work with further investigation.
The paper tackled the problem of cloud coverage in remote sensing imagery by treating clouds as inherent noise and using a convolutional LSTM network for vegetation classification without explicit cloud filtering, achieving state-of-the-art classification accuracies and demonstrating internalized cloud-filtering mechanisms through visualizations and ablation experiments.
Clouds frequently cover the Earth's surface and pose an omnipresent challenge to optical Earth observation methods. The vast majority of remote sensing approaches either selectively choose single cloud-free observations or employ a pre-classification strategy to identify and mask cloudy pixels. We follow a different strategy and treat cloud coverage as noise that is inherent to the observed satellite data. In prior work, we directly employed a straightforward \emph{convolutional long short-term memory} network for vegetation classification without explicit cloud filtering and achieved state-of-the-art classification accuracies. In this work, we investigate this cloud-robustness further by visualizing internal cell activations and performing an ablation experiment on datasets of different cloud coverage. In the visualizations of network states, we identified some cells in which modulation and input gates closed on cloudy pixels. This indicates that the network has internalized a cloud-filtering mechanism without being specifically trained on cloud labels. Overall, our results question the necessity of sophisticated pre-processing pipelines for multi-temporal deep learning approaches.