CAIRNS: Balancing Readability and Scientific Accuracy in Climate Adaptation Question Answering
This addresses the need for credible and accessible information for farmer advisors in agriculture, though it appears incremental as it builds on existing methods without fine-tuning or reinforcement learning.
The paper tackles the problem of providing readable and scientifically accurate answers for climate adaptation questions by introducing CAIRNS, a framework that improves readability and citation reliability, and it outperforms baselines on most metrics in expert-curated evaluations.
Climate adaptation strategies are proposed in response to climate change. They are practised in agriculture to sustain food production. These strategies can be found in unstructured data (for example, scientific literature from the Elsevier website) or structured (heterogeneous climate data via government APIs). We present Climate Adaptation question-answering with Improved Readability and Noted Sources (CAIRNS), a framework that enables experts -- farmer advisors -- to obtain credible preliminary answers from complex evidence sources from the web. It enhances readability and citation reliability through a structured ScholarGuide prompt and achieves robust evaluation via a consistency-weighted hybrid evaluator that leverages inter-model agreement with experts. Together, these components enable readable, verifiable, and domain-grounded question-answering without fine-tuning or reinforcement learning. Using a previously reported dataset of expert-curated question-answers, we show that CAIRNS outperforms the baselines on most of the metrics. Our thorough ablation study confirms the results on all metrics. To validate our LLM-based evaluation, we also report an analysis of correlations against human judgment.