Ask to Know More: Generating Counterfactual Explanations for Fake Claims
This addresses the need for interpretability in fact-checking systems to help people understand predictions, though it is incremental as it builds on existing datasets and methods.
The paper tackles the problem of making automated fact-checking systems more understandable by generating counterfactual explanations for fake news claims, using a method based on question answering and entailment reasoning, and shows it produces more helpful explanations than state-of-the-art methods in evaluations on the FEVER dataset.
Automated fact checking systems have been proposed that quickly provide veracity prediction at scale to mitigate the negative influence of fake news on people and on public opinion. However, most studies focus on veracity classifiers of those systems, which merely predict the truthfulness of news articles. We posit that effective fact checking also relies on people's understanding of the predictions. In this paper, we propose elucidating fact checking predictions using counterfactual explanations to help people understand why a specific piece of news was identified as fake. In this work, generating counterfactual explanations for fake news involves three steps: asking good questions, finding contradictions, and reasoning appropriately. We frame this research question as contradicted entailment reasoning through question answering (QA). We first ask questions towards the false claim and retrieve potential answers from the relevant evidence documents. Then, we identify the most contradictory answer to the false claim by use of an entailment classifier. Finally, a counterfactual explanation is created using a matched QA pair with three different counterfactual explanation forms. Experiments are conducted on the FEVER dataset for both system and human evaluations. Results suggest that the proposed approach generates the most helpful explanations compared to state-of-the-art methods.