Current Trends in Deep Learning for Earth Observation: An Open-source Benchmark Arena for Image Classification
This work provides a standardized benchmark for researchers and practitioners in Earth Observation to compare deep learning methods, but it is incremental as it aggregates existing models and datasets without proposing new algorithms.
The authors introduced AiTLAS: Benchmark Arena, an open-source benchmark suite for evaluating over 500 deep learning models from ten architectures on 22 Earth Observation datasets for image classification, including transfer learning approaches, with all resources publicly available for reproducibility.
We present AiTLAS: Benchmark Arena -- an open-source benchmark suite for evaluating state-of-the-art deep learning approaches for image classification in Earth Observation (EO). To this end, we present a comprehensive comparative analysis of more than 500 models derived from ten different state-of-the-art architectures and compare them to a variety of multi-class and multi-label classification tasks from 22 datasets with different sizes and properties. In addition to models trained entirely on these datasets, we benchmark models trained in the context of transfer learning, leveraging pre-trained model variants, as it is typically performed in practice. All presented approaches are general and can be easily extended to many other remote sensing image classification tasks not considered in this study. To ensure reproducibility and facilitate better usability and further developments, all of the experimental resources including the trained models, model configurations, and processing details of the datasets (with their corresponding splits used for training and evaluating the models) are publicly available on the repository: https://github.com/biasvariancelabs/aitlas-arena