Multiresolution Fully Convolutional Networks to detect Clouds and Snow through Optical Satellite Images
This work addresses a domain-specific challenge in remote sensing for applications like cloud detection, though it is incremental as it builds on existing fusion and FCN methods.
The paper tackled the problem of distinguishing clouds from snow in high-resolution optical satellite images by introducing a multiresolution fully convolutional network that fuses visible/near-infrared and shortwave-infrared bands, achieving 94.31% overall accuracy and a 97.67% F1 score for clouds, with improvements of 30% over Random Forest and 10% over a single-resolution FCN.
Clouds and snow have similar spectral features in the visible and near-infrared (VNIR) range and are thus difficult to distinguish from each other in high resolution VNIR images. We address this issue by introducing a shortwave-infrared (SWIR) band where clouds are highly reflective, and snow is absorptive. As SWIR is typically of a lower resolution compared to VNIR, this study proposes a multiresolution fully convolutional neural network (FCN) that can effectively detect clouds and snow in VNIR images. We fuse the multiresolution bands within a deep FCN and perform semantic segmentation at the higher, VNIR resolution. Such a fusion-based classifier, trained in an end-to-end manner, achieved 94.31% overall accuracy and an F1 score of 97.67% for clouds on Resourcesat-2 data captured over the state of Uttarakhand, India. These scores were found to be 30% higher than a Random Forest classifier, and 10% higher than a standalone single-resolution FCN. Apart from being useful for cloud detection purposes, the study also highlights the potential of convolutional neural networks for multi-sensor fusion problems.