SenPa-MAE: Sensor Parameter Aware Masked Autoencoder for Multi-Satellite Self-Supervised Pretraining
This addresses the challenge of sensor diversity in Earth observation for researchers and practitioners, offering a step towards a sensor-independent foundation model, though it is incremental in building on existing masked autoencoder methods.
The paper tackles the problem of pre-training models on multi-satellite imagery with diverse sensor characteristics by introducing SenPa-MAE, which encodes sensor parameters into image embeddings, enabling effective differentiation between sensors and understanding of their correlations to observed signals.
This paper introduces SenPa-MAE, a transformer architecture that encodes the sensor parameters of an observed multispectral signal into the image embeddings. SenPa-MAE can be pre-trained on imagery of different satellites with non-matching spectral or geometrical sensor characteristics. To incorporate sensor parameters, we propose a versatile sensor parameter encoding module as well as a data augmentation strategy for the diversification of the pre-training dataset. This enables the model to effectively differentiate between various sensors and gain an understanding of sensor parameters and the correlation to the observed signal. Given the rising number of Earth observation satellite missions and the diversity in their sensor specifications, our approach paves the way towards a sensor-independent Earth observation foundation model. This opens up possibilities such as cross-sensor training and sensor-independent inference.