Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensing
This work addresses a domain-specific issue in remote sensing by enhancing self-supervised learning for noisy SAR and multispectral images, though it is incremental as it builds on existing MAE methods.
The paper tackles the problem of masked autoencoders (MAE) focusing too much on pixel details in remote sensing, which limits semantic understanding, especially for noisy SAR images, by proposing FG-MAE that reconstructs features like HOG and NDI instead of raw pixels, resulting in improved performance on downstream tasks with a particular boost for SAR imagery.
Self-supervised learning guided by masked image modelling, such as Masked AutoEncoder (MAE), has attracted wide attention for pretraining vision transformers in remote sensing. However, MAE tends to excessively focus on pixel details, thereby limiting the model's capacity for semantic understanding, in particular for noisy SAR images. In this paper, we explore spectral and spatial remote sensing image features as improved MAE-reconstruction targets. We first conduct a study on reconstructing various image features, all performing comparably well or better than raw pixels. Based on such observations, we propose Feature Guided Masked Autoencoder (FG-MAE): reconstructing a combination of Histograms of Oriented Graidents (HOG) and Normalized Difference Indices (NDI) for multispectral images, and reconstructing HOG for SAR images. Experimental results on three downstream tasks illustrate the effectiveness of FG-MAE with a particular boost for SAR imagery. Furthermore, we demonstrate the well-inherited scalability of FG-MAE and release a first series of pretrained vision transformers for medium resolution SAR and multispectral images.