The Intended Uses of Automated Fact-Checking Artefacts: Why, How and Who
This work addresses a critical gap in automated fact-checking research by highlighting how vague claims impede progress and offering recommendations for clearer communication, though it is incremental in its analysis.
The paper analyzed 100 highly-cited automated fact-checking papers and found that many lack clarity in describing intended uses, propose inconsistent strategies, and lack empirical backing, hindering the technology's effectiveness.
Automated fact-checking is often presented as an epistemic tool that fact-checkers, social media consumers, and other stakeholders can use to fight misinformation. Nevertheless, few papers thoroughly discuss how. We document this by analysing 100 highly-cited papers, and annotating epistemic elements related to intended use, i.e., means, ends, and stakeholders. We find that narratives leaving out some of these aspects are common, that many papers propose inconsistent means and ends, and that the feasibility of suggested strategies rarely has empirical backing. We argue that this vagueness actively hinders the technology from reaching its goals, as it encourages overclaiming, limits criticism, and prevents stakeholder feedback. Accordingly, we provide several recommendations for thinking and writing about the use of fact-checking artefacts.