Mitigation of Diachronic Bias in Fake News Detection Dataset
This addresses a specific bias issue in fake news detection for researchers and practitioners, but it is incremental as it builds on existing dataset and method improvements.
The study tackled the problem of diachronic bias in fake news detection datasets, where models trained on time-specific data struggle with novel fake news due to changes in proper nouns like person names, and proposed masking methods using Wikidata to mitigate this bias, showing improved robustness in experiments.
Fake news causes significant damage to society.To deal with these fake news, several studies on building detection models and arranging datasets have been conducted. Most of the fake news datasets depend on a specific time period. Consequently, the detection models trained on such a dataset have difficulty detecting novel fake news generated by political changes and social changes; they may possibly result in biased output from the input, including specific person names and organizational names. We refer to this problem as \textbf{Diachronic Bias} because it is caused by the creation date of news in each dataset. In this study, we confirm the bias, especially proper nouns including person names, from the deviation of phrase appearances in each dataset. Based on these findings, we propose masking methods using Wikidata to mitigate the influence of person names and validate whether they make fake news detection models robust through experiments with in-domain and out-of-domain data.