Beyond the "Truth": Investigating Election Rumors on Truth Social During the 2024 Election
This work addresses misinformation spread in ideologically homogeneous networks, providing large-scale empirical evidence for psychological phenomena, though it is incremental in applying existing methods to a new platform.
This paper tackled the problem of analyzing election rumors on Truth Social during the 2024 election by compiling a large-scale dataset, developing a multistage Rumor Detection Agent using LLMs, and quantifying psychological dynamics like the illusory truth effect, finding that sharing probability rises with exposure and nearly one quarter of users become 'infected' within four propagation iterations.
Large language models (LLMs) offer unprecedented opportunities for analyzing social phenomena at scale. This paper demonstrates the value of LLMs in psychological measurement by (1) compiling the first large-scale dataset of election rumors on a niche alt-tech platform, (2) developing a multistage Rumor Detection Agent that leverages LLMs for high-precision content classification, and (3) quantifying the psychological dynamics of rumor propagation, specifically the "illusory truth effect" in a naturalistic setting. The Rumor Detection Agent combines (i) a synthetic data-augmented, fine-tuned RoBERTa classifier, (ii) precision keyword filtering, and (iii) a two-pass LLM verification pipeline using GPT-4o mini. The findings reveal that sharing probability rises steadily with each additional exposure, providing large-scale empirical evidence for dose-response belief reinforcement in ideologically homogeneous networks. Simulation results further demonstrate rapid contagion effects: nearly one quarter of users become "infected" within just four propagation iterations. Taken together, these results illustrate how LLMs can transform psychological science by enabling the rigorous measurement of belief dynamics and misinformation spread in massive, real-world datasets.