A Multi-Label Dataset of French Fake News: Human and Machine Insights
This work provides a dataset and analysis for fake news detection in French press, but it is incremental as it builds on existing annotation and classification methods.
The researchers tackled the problem of identifying fake news by creating OBSINFOX, a multi-label dataset of 100 French documents from unreliable sources, annotated with 11 labels by 8 annotators, and used it to analyze human and machine insights, such as linking subjectivity to fake news ascriptions.
We present a corpus of 100 documents, OBSINFOX, selected from 17 sources of French press considered unreliable by expert agencies, annotated using 11 labels by 8 annotators. By collecting more labels than usual, by more annotators than is typically done, we can identify features that humans consider as characteristic of fake news, and compare them to the predictions of automated classifiers. We present a topic and genre analysis using Gate Cloud, indicative of the prevalence of satire-like text in the corpus. We then use the subjectivity analyzer VAGO, and a neural version of it, to clarify the link between ascriptions of the label Subjective and ascriptions of the label Fake News. The annotated dataset is available online at the following url: https://github.com/obs-info/obsinfox Keywords: Fake News, Multi-Labels, Subjectivity, Vagueness, Detail, Opinion, Exaggeration, French Press