Continual Vision-Language Learning for Remote Sensing: Benchmarking and Analysis
This work addresses the challenge of enabling RS VLMs to adapt continuously to new data and tasks without forgetting, which is crucial for practical applications in remote sensing, but it is incremental as it focuses on benchmarking and analysis rather than proposing a new solution.
The authors tackled the problem of catastrophic forgetting in remote sensing vision-language models (RS VLMs) by creating CLeaRS, a benchmark with over 207k image-text pairs across 10 subsets, and found that current models and adapted continual learning methods fail to effectively handle continual adaptation in various settings.
Current remote sensing vision-language models (RS VLMs) demonstrate impressive performance in image interpretation but rely on static training data, limiting their ability to accommodate continuously emerging sensing modalities and downstream tasks. This exposes a fundamental challenge: enabling RS VLMs to continually adapt without catastrophic forgetting. Despite its practical importance, the continual learning capability of RS VLMs remains underexplored, and no dedicated benchmark currently exists. In this work, we present CLeaRS, a comprehensive benchmark for continual vision-language learning in remote sensing. CLeaRS comprises 10 curated subsets with over 207k image-text pairs, spanning diverse interpretation tasks, sensing modalities, and application scenarios. We further define three evaluation protocols: long-horizon, modality-incremental, and task-incremental settings, to systematically assess continual adaptation. Extensive benchmarking of diverse vision-language models reveals catastrophic forgetting across all settings. Moreover, representative continual learning methods, when adapted to RS VLMs, exhibit limited effectiveness in handling task, instruction, and modality transitions. Our findings underscore the need for developing continual learning methods tailored to RS VLMs.