Computational Fact-Checking of Online Discourse: Scoring scientific accuracy in climate change related news articles
This work addresses misinformation in civic discourse for democratic societies, but it is incremental as it builds on existing methods and highlights limitations in scalability.
The paper tackles the problem of misinformation in climate change news by developing a semi-automatic system to quantify scientific accuracy using LLM-based statement extraction and knowledge graph analysis, which streamlines veracity quantification but lacks granularity and scale for public media annotation.
Democratic societies need reliable information. Misinformation in popular media such as news articles or videos threatens to impair civic discourse. Citizens are, unfortunately, not equipped to verify this content flood consumed daily at increasing rates. This work aims to semi-automatically quantify scientific accuracy of online media. By semantifying media of unknown veracity, their statements can be compared against equally processed trusted sources. We implemented a workflow using LLM-based statement extraction and knowledge graph analysis. Our neurosymbolic system was able to evidently streamline state-of-the-art veracity quantification. Evaluated via expert interviews and a user survey, the tool provides a beneficial veracity indication. This indicator, however, is unable to annotate public media at the required granularity and scale. Further work towards a FAIR (Findable, Accessible, Interoperable, Reusable) ground truth and complementary metrics are required to scientifically support civic discourse.