How Good Are SOTA Fake News Detectors
This addresses the problem of unreliable fake news detection for social media platforms and users, but it is incremental as it focuses on benchmarking existing methods.
The paper evaluated the robustness of state-of-the-art fake news detectors, finding that traditional models generalize better to out-of-distribution data than large language models, with performance depending on the specific task.
Automatic fake news detection with machine learning can prevent the dissemination of false statements before they gain many views. Several datasets labeling statements as legitimate or false have been created since the 2016 United States presidential election for the prospect of training machine learning models. We evaluate the robustness of both traditional and deep state-of-the-art models to gauge how well they may perform in the real world. We find that traditional models tend to generalize better to data outside the distribution it was trained on compared to more recently-developed large language models, though the best model to use may depend on the specific task at hand.