PHEMEPlus: Enriching Social Media Rumour Verification with External Evidence
This addresses the lack of datasets combining social media and external evidence for rumour verification, though it is incremental as an extension of the PHEME benchmark.
The authors tackled the problem of social media rumour verification by creating PHEMEPlus, a dataset that combines social media conversations with external web evidence, and demonstrated that incorporating this evidence improves verification models with concrete performance gains.
Work on social media rumour verification utilises signals from posts, their propagation and users involved. Other lines of work target identifying and fact-checking claims based on information from Wikipedia, or trustworthy news articles without considering social media context. However works combining the information from social media with external evidence from the wider web are lacking. To facilitate research in this direction, we release a novel dataset, PHEMEPlus, an extension of the PHEME benchmark, which contains social media conversations as well as relevant external evidence for each rumour. We demonstrate the effectiveness of incorporating such evidence in improving rumour verification models. Additionally, as part of the evidence collection, we evaluate various ways of query formulation to identify the most effective method.