AI Propaganda factories with language models
This research addresses the threat of automated propaganda for society and security, highlighting an incremental advancement in understanding AI-driven influence operations.
The study tackled the problem of AI-powered influence operations by demonstrating that small language models can produce coherent, persona-driven political messaging, with findings showing that persona design explains behavior more than model identity and that engagement as a stressor increases ideological adherence and extreme content. The result indicates that fully automated influence-content production is accessible to various actors, suggesting a shift in defense strategies towards detection and disruption.
AI-powered influence operations can now be executed end-to-end on commodity hardware. We show that small language models produce coherent, persona-driven political messaging and can be evaluated automatically without human raters. Two behavioural findings emerge. First, persona-over-model: persona design explains behaviour more than model identity. Second, engagement as a stressor: when replies must counter-arguments, ideological adherence strengthens and the prevalence of extreme content increases. We demonstrate that fully automated influence-content production is within reach of both large and small actors. Consequently, defence should shift from restricting model access towards conversation-centric detection and disruption of campaigns and coordination infrastructure. Paradoxically, the very consistency that enables these operations also provides a detection signature.