Are LLM-Powered Social Media Bots Realistic?
This work addresses the potential for LLM-powered bots in social media, with implications for detection and effectiveness, but it is incremental as it builds on existing network and linguistic analysis methods.
The study investigated the realism of social media bots powered by Large Language Models (LLMs) by creating synthetic bot networks and comparing them to empirical data, finding that both network and linguistic properties of LLM-powered bots differ from wild bots and humans.
As Large Language Models (LLMs) become more sophisticated, there is a possibility to harness LLMs to power social media bots. This work investigates the realism of generating LLM-Powered social media bot networks. Through a combination of manual effort, network science and LLMs, we create synthetic bot agent personas, their tweets and their interactions, thereby simulating social media networks. We compare the generated networks against empirical bot/human data, observing that both network and linguistic properties of LLM-Powered Bots differ from Wild Bots/Humans. This has implications towards the detection and effectiveness of LLM-Powered Bots.