NeuCo-Bench: A Novel Benchmark Framework for Neural Embeddings in Earth Observation
This addresses the problem of inconsistent benchmarking for neural embeddings in Earth Observation, offering a community-driven tool for researchers, though it is incremental as it builds on existing embedding and evaluation concepts.
The authors tackled the lack of standardized evaluation for neural embeddings in Earth Observation by introducing NeuCo-Bench, a benchmark framework with components like reusable embeddings and a hidden-task leaderboard, resulting in initial results from a public challenge and ablations with state-of-the-art models.
We introduce NeuCo-Bench, a novel benchmark framework for evaluating (lossy) neural compression and representation learning in the context of Earth Observation (EO). Our approach builds on fixed-size embeddings that act as compact, task-agnostic representations applicable to a broad range of downstream tasks. NeuCo-Bench comprises three core components: (i) an evaluation pipeline built around reusable embeddings, (ii) a new challenge mode with a hidden-task leaderboard designed to mitigate pretraining bias, and (iii) a scoring system that balances accuracy and stability. To support reproducibility, we release SSL4EO-S12-downstream, a curated multispectral, multitemporal EO dataset. We present initial results from a public challenge at the 2025 CVPR EARTHVISION workshop and conduct ablations with state-of-the-art foundation models. NeuCo-Bench provides a first step towards community-driven, standardized evaluation of neural embeddings for EO and beyond.