BotaCLIP: Contrastive Learning for Botany-Aware Representation of Earth Observation Data
This work addresses the problem of efficiently injecting expert knowledge into foundation models for biodiversity modeling in data-scarce settings, representing an incremental advancement in domain-specific adaptation.
The paper tackled the challenge of adapting a pre-trained Earth Observation foundation model to incorporate domain-specific botanical knowledge without extensive retraining, resulting in BotaCLIP, which improved performance in ecological tasks like plant presence prediction and butterfly occurrence modeling over existing methods.
Foundation models have demonstrated a remarkable ability to learn rich, transferable representations across diverse modalities such as images, text, and audio. In modern machine learning pipelines, these representations often replace raw data as the primary input for downstream tasks. In this paper, we address the challenge of adapting a pre-trained foundation model to inject domain-specific knowledge, without retraining from scratch or incurring significant computational costs. To this end, we introduce BotaCLIP, a lightweight multimodal contrastive framework that adapts a pre-trained Earth Observation foundation model (DOFA) by aligning high-resolution aerial imagery with botanical relevés. Unlike generic embeddings, BotaCLIP internalizes ecological structure through contrastive learning with a regularization strategy that mitigates catastrophic forgetting. Once trained, the resulting embeddings serve as transferable representations for downstream predictors. Motivated by real-world applications in biodiversity modeling, we evaluated BotaCLIP representations in three ecological tasks: plant presence prediction, butterfly occurrence modeling, and soil trophic group abundance estimation. The results showed consistent improvements over those derived from DOFA and supervised baselines. More broadly, this work illustrates how domain-aware adaptation of foundation models can inject expert knowledge into data-scarce settings, enabling frugal representation learning.