FakeGPT: Fake News Generation, Explanation and Detection of Large Language Models
This addresses the societal problem of fake news spread by evaluating LLMs, but it is incremental as it applies existing methods to new tasks without major breakthroughs.
The study assessed ChatGPT's ability to generate, explain, and detect fake news, finding it produces high-quality fake news samples and shows commendable but improvable detection performance, with specific improvements through a reason-aware prompt method.
The rampant spread of fake news has adversely affected society, resulting in extensive research on curbing its spread. As a notable milestone in large language models (LLMs), ChatGPT has gained significant attention due to its exceptional natural language processing capabilities. In this study, we present a thorough exploration of ChatGPT's proficiency in generating, explaining, and detecting fake news as follows. Generation -- We employ four prompt methods to generate fake news samples and prove the high quality of these samples through both self-assessment and human evaluation. Explanation -- We obtain nine features to characterize fake news based on ChatGPT's explanations and analyze the distribution of these factors across multiple public datasets. Detection -- We examine ChatGPT's capacity to identify fake news. We explore its detection consistency and then propose a reason-aware prompt method to improve its performance. Although our experiments demonstrate that ChatGPT shows commendable performance in detecting fake news, there is still room for its improvement. Consequently, we further probe into the potential extra information that could bolster its effectiveness in detecting fake news.