SLURG: Investigating the Feasibility of Generating Synthetic Online Fallacious Discourse
This work addresses the challenge of detecting misinformation in social media forums, particularly for the Ukrainian-Russian conflict domain, but is incremental as it builds on existing fallacy detection methods.
The paper tackled the problem of detecting manipulation in online discourse by exploring the generation of synthetic fallacious comments using large language models, finding that LLMs can replicate syntactic patterns and that few-shot prompts improve vocabulary diversity.
In our paper we explore the definition, and extrapolation of fallacies as they pertain to the automatic detection of manipulation on social media. In particular we explore how these logical fallacies might appear in the real world i.e internet forums. We discovered a prevalence of misinformation / misguided intention in discussion boards specifically centered around the Ukrainian Russian Conflict which serves to narrow the domain of our task. Although automatic fallacy detection has gained attention recently, most datasets use unregulated fallacy taxonomies or are limited to formal linguistic domains like political debates or news reports. Online discourse, however, often features non-standardized and diverse language not captured in these domains. We present Shady Linguistic Utterance Replication-Generation (SLURG) to address these limitations, exploring the feasibility of generating synthetic fallacious forum-style comments using large language models (LLMs), specifically DeepHermes-3-Mistral-24B. Our findings indicate that LLMs can replicate the syntactic patterns of real data} and that high-quality few-shot prompts enhance LLMs' ability to mimic the vocabulary diversity of online forums.