What Transfers Under Source Shift? Definitions, Examples, and Fine-Tuning for Climate Disclosure Classification
For practitioners applying LLMs to climate disclosure classification across diverse sources, the paper provides guidance on which adaptation strategies are robust to source shift.
The paper studies cross-source adaptation for climate disclosure classification, finding that simpler strategies like definitions and random few-shot examples transfer more reliably under source shift than complex methods like similarity-based retrieval or LoRA fine-tuning, which lose their in-source advantage.
Climate disclosure classification is a fundamental task for analysing corporate climate disclosures, yet such disclosures appear in many different sources -- annual reports, press releases, and earnings calls -- that differ in length, purpose, and writing style. Existing evaluations are mostly conducted within a single source, leaving open whether common LLM adaptation strategies remain effective under source shift. We reframe climate disclosure classification as a cross-source adaptation problem and study three widely used adaptation strategies -- definitions, examples, and fine-tuning -- across eleven open- and closed-source LLMs, using two corpora that share the same label space but come from different sources. We find that all strategies bring positive cross-source gains on average, but the strongest in-source strategies are not the strongest cross-source ones: similarity-based retrieval and LoRA fine-tuning gain most in-source but lose most of that advantage under source shift; randomly selected few-shot examples, a weaker in-source baseline, retain their advantage more reliably; definitions transfer most consistently, though only when their granularity matches the target text. Across these strategies, when the source changes, simpler is often safer.