How Much Does Persuasion Strategy Matter? LLM-Annotated Evidence from Charitable Donation Dialogues
This work addresses the problem of understanding effective persuasion strategies for charitable fundraising, though it is incremental in applying LLM annotation to an existing dataset.
The study investigated which persuasion strategies affect charitable donation compliance by annotating 10,600 persuader turns with 41 strategies using three LLMs, finding that strategy categories explained little variance in outcomes (pseudo R² ≈ 0.015), with Guilt Induction significantly reducing donation rates by about 23 percentage points while Reciprocity was the most robust positive correlate.
Which persuasion strategies, if any, are associated with donation compliance? Answering this requires fine-grained strategy labels across a full corpus and statistical tests corrected for multiple comparisons. We annotate all 10,600 persuader turns in the 1,017-dialogue PersuasionForGood corpus (Wang et al., 2019), where donation outcomes are directly observable, with a taxonomy of 41 strategies in 11 categories, using three open-source large language models (LLMs; Qwen3:30b, Mistral-Small-3.2, Phi-4). Strategy categories alone explain little variance in donation outcome (pseudo $R^2 \approx 0.015$, consistent across all three annotators). Guilt Induction is the only strategy significantly associated with lower donation rates ($Δ\approx -23$ percentage points), an effect that replicates across all three models despite only moderate inter-model agreement. Reciprocity is the most robust positive correlate. Target sentiment and interest predict whether a donation occurs but show at most a weak correlation with donation amount. These findings suggest that strategy identification alone is insufficient to explain persuasion effectiveness, and that guilt-based appeals may be counterproductive in prosocial settings. We release the fully annotated corpus as a public resource.