Detecting Winning Arguments with Large Language Models and Persuasion Strategies
This work addresses the problem of understanding human communication for researchers and practitioners in NLP, but it is incremental as it builds on existing methods with structured prompting.
The paper tackled detecting persuasion in argumentative text by investigating persuasion strategies like Attack on reputation, using large language models with a Multi-Strategy Persuasion Scoring approach on three datasets including Winning Arguments from Change My View. Results showed that strategy-guided reasoning improved persuasiveness prediction, with the topic-annotated dataset released publicly.
Detecting persuasion in argumentative text is a challenging task with important implications for understanding human communication. This work investigates the role of persuasion strategies - such as Attack on reputation, Distraction, and Manipulative wording - in determining the persuasiveness of a text. We conduct experiments on three annotated argument datasets: Winning Arguments (built from the Change My View subreddit), Anthropic/Persuasion, and Persuasion for Good. Our approach leverages large language models (LLMs) with a Multi-Strategy Persuasion Scoring approach that guides reasoning over six persuasion strategies. Results show that strategy-guided reasoning improves the prediction of persuasiveness. To better understand the influence of content, we organize the Winning Argument dataset into broad discussion topics and analyze performance across them. We publicly release this topic-annotated version of the dataset to facilitate future research. Overall, our methodology demonstrates the value of structured, strategy-aware prompting for enhancing interpretability and robustness in argument quality assessment.