Exploring the Deceptive Power of LLM-Generated Fake News: A Study of Real-World Detection Challenges
This work addresses the challenge of fake news detection in complex domains like healthcare, but it is incremental as it builds on existing LLM-based attacks by improving prompting techniques.
This paper tackles the problem of detecting LLM-generated fake news, particularly in healthcare, by proposing a new attack method called VLPrompt that eliminates human intervention and maintains context, and it introduces a dataset (VLPFN) for evaluation, with experiments showing performance metrics on detection methods.
Recent advancements in Large Language Models (LLMs) have enabled the creation of fake news, particularly in complex fields like healthcare. Studies highlight the gap in the deceptive power of LLM-generated fake news with and without human assistance, yet the potential of prompting techniques has not been fully explored. Thus, this work aims to determine whether prompting strategies can effectively narrow this gap. Current LLM-based fake news attacks require human intervention for information gathering and often miss details and fail to maintain context consistency. Therefore, to better understand threat tactics, we propose a strong fake news attack method called conditional Variational-autoencoder-Like Prompt (VLPrompt). Unlike current methods, VLPrompt eliminates the need for additional data collection while maintaining contextual coherence and preserving the intricacies of the original text. To propel future research on detecting VLPrompt attacks, we created a new dataset named VLPrompt fake news (VLPFN) containing real and fake texts. Our experiments, including various detection methods and novel human study metrics, were conducted to assess their performance on our dataset, yielding numerous findings.