The Anatomy of Speech Persuasion: Linguistic Shifts in LLM-Modified Speeches
This addresses the problem of interpreting LLM behavior in persuasion tasks for researchers, but it is incremental as it builds on existing datasets and models.
This study tackled how large language models understand persuasiveness by modifying speech transcripts from a French competition, finding that GPT-4o applies systematic stylistic modifications like manipulating emotional lexicon and syntactic structures rather than optimizing persuasiveness in a human-like manner.
This study examines how large language models understand the concept of persuasiveness in public speaking by modifying speech transcripts from PhD candidates in the "Ma These en 180 Secondes" competition, using the 3MT French dataset. Our contributions include a novel methodology and an interpretable textual feature set integrating rhetorical devices and discourse markers. We prompt GPT-4o to enhance or diminish persuasiveness and analyze linguistic shifts between original and generated speech in terms of the new features. Results indicate that GPT-4o applies systematic stylistic modifications rather than optimizing persuasiveness in a human-like manner. Notably, it manipulates emotional lexicon and syntactic structures (such as interrogative and exclamatory clauses) to amplify rhetorical impact.