The View From Space: Navigating Instrumentation Differences with EOFMs
This addresses a critical issue for remote sensing scientists and model developers by highlighting pitfalls in current EOFM design, though it is incremental as it builds on existing work without introducing a new method.
The study tackled the problem of Earth Observation Foundation Models (EOFMs) being sensitive to sensor architecture differences, showing that their representation spaces are highly affected, which impacts their robustness and application across diverse data modalities.
Earth Observation Foundation Models (EOFMs) have exploded in prevalence as tools for processing the massive volumes of remotely sensed and other earth observation data, and for delivering impact on the many essential earth monitoring tasks. An emerging trend posits using the outputs of pre-trained models as 'embeddings' which summarize high dimensional data to be used for generic tasks such as similarity search and content-specific queries. However, most EOFM models are trained only on single modalities of data and then applied or benchmarked by matching bands across different modalities. It is not clear from existing work what impact diverse sensor architectures have on the internal representations of the present suite of EOFMs. We show in this work that the representation space of EOFMs is highly sensitive to sensor architecture and that understanding this difference gives a vital perspective on the pitfalls of current EOFM design and signals for how to move forward as model developers, users, and a community guided by robust remote-sensing science.