Crafting Tomorrow's Headlines: Neural News Generation and Detection in English, Turkish, Hungarian, and Persian
This work addresses the threat of misinformation from LLMs by providing a multilingual benchmark for news detection, though it is incremental as it builds on existing detection methods.
The paper tackled the problem of detecting machine-generated news by creating a benchmark dataset in English, Turkish, Hungarian, and Persian, and found that detection performance varied across languages and models, with some classifiers achieving high accuracy but facing challenges in robustness and interpretability.
In the era dominated by information overload and its facilitation with Large Language Models (LLMs), the prevalence of misinformation poses a significant threat to public discourse and societal well-being. A critical concern at present involves the identification of machine-generated news. In this work, we take a significant step by introducing a benchmark dataset designed for neural news detection in four languages: English, Turkish, Hungarian, and Persian. The dataset incorporates outputs from multiple multilingual generators (in both, zero-shot and fine-tuned setups) such as BloomZ, LLaMa-2, Mistral, Mixtral, and GPT-4. Next, we experiment with a variety of classifiers, ranging from those based on linguistic features to advanced Transformer-based models and LLMs prompting. We present the detection results aiming to delve into the interpretablity and robustness of machine-generated texts detectors across all target languages.