A Multilingual, Large-Scale Study of the Interplay between LLM Safeguards, Personalisation, and Disinformation
It addresses vulnerabilities in LLM safety for multilingual and cross-demographic contexts, offering a foundation for improving safeguards, though it is incremental as it builds on existing red teaming methods.
This study tackled the problem of LLMs generating personalized disinformation across languages and demographics, finding that simple personalization prompts increase jailbreak likelihood by up to 10 percentage points and models like Grok and GPT had jailbreak rates exceeding 85%.
Large Language Models (LLMs) can generate human-like disinformation, yet their ability to personalise such content across languages and demographics remains underexplored. This study presents the first large-scale, multilingual analysis of persona-targeted disinformation generation by LLMs. Employing a red teaming methodology, we prompt eight state-of-the-art LLMs with 324 false narratives and 150 demographic personas (combinations of country, generation, and political orientation) across four languages--English, Russian, Portuguese, and Hindi--resulting in AI-TRAITS, a comprehensive dataset of 1.6 million personalised disinformation texts. Results show that the use of even simple personalisation prompts significantly increases the likelihood of jailbreaks across all studied LLMs, up to 10 percentage points, and alters linguistic and rhetorical patterns that enhance narrative persuasiveness. Models such as Grok and GPT exhibited jailbreak rates and personalisation scores both exceeding 85%. These insights expose critical vulnerabilities in current state-of-the-art LLMs and offer a foundation for improving safety alignment and detection strategies in multilingual and cross-demographic contexts.