AGI for the Earth, the path, possibilities and how to evaluate intelligence of models that work with Earth Observation Data?
This work addresses the need for better evaluation benchmarks for AGI models using Earth Observation data, which is incremental as it builds on existing efforts by highlighting limitations and proposing new tasks.
The paper argues that Earth Observation data is a valuable but underutilized modality for advancing Artificial General Intelligence, and it proposes a comprehensive set of tasks to create a benchmark for evaluating the generalization ability of foundation models in this domain.
Artificial General Intelligence (AGI) is closer than ever to becoming a reality, sparking widespread enthusiasm in the research community to collect and work with various modalities, including text, image, video, and audio. Despite recent efforts, satellite spectral imagery, as an additional modality, has yet to receive the attention it deserves. This area presents unique challenges, but also holds great promise in advancing the capabilities of AGI in understanding the natural world. In this paper, we argue why Earth Observation data is useful for an intelligent model, and then we review existing benchmarks and highlight their limitations in evaluating the generalization ability of foundation models in this domain. This paper emphasizes the need for a more comprehensive benchmark to evaluate earth observation models. To facilitate this, we propose a comprehensive set of tasks that a benchmark should encompass to effectively assess a model's ability to understand and interact with Earth observation data.