Automated Fact-Checking of Climate Change Claims with Large Language Models
This addresses the problem of misinformation in climate science for the public and researchers, though it is incremental as it builds on existing LLM and fact-checking methods.
The paper tackles automated fact-checking of climate change claims by introducing Climinator, a tool using Large Language Models and a Mediator-Advocate framework, which achieves high accuracy on datasets from Climate Feedback and Skeptical Science and reliably converges to scientific consensus even when including denial perspectives.
This paper presents Climinator, a novel AI-based tool designed to automate the fact-checking of climate change claims. Utilizing an array of Large Language Models (LLMs) informed by authoritative sources like the IPCC reports and peer-reviewed scientific literature, Climinator employs an innovative Mediator-Advocate framework. This design allows Climinator to effectively synthesize varying scientific perspectives, leading to robust, evidence-based evaluations. Our model demonstrates remarkable accuracy when testing claims collected from Climate Feedback and Skeptical Science. Notably, when integrating an advocate with a climate science denial perspective in our framework, Climinator's iterative debate process reliably converges towards scientific consensus, underscoring its adeptness at reconciling diverse viewpoints into science-based, factual conclusions. While our research is subject to certain limitations and necessitates careful interpretation, our approach holds significant potential. We hope to stimulate further research and encourage exploring its applicability in other contexts, including political fact-checking and legal domains.