EmoDistill: Offline Emotion Skill Distillation for Language Model Agents in Adversarial Negotiation
For developers of LLM-based agents in adversarial settings, this work addresses the vulnerability of aligned models to emotional manipulation and provides a method to strategically use emotion as a negotiation tool.
The paper shows that emotional framing in adversarial negotiation can steer LLM agents toward counterparty interests, and introduces EmoDistill, an offline framework that distills emotional negotiation skills into small language models via Implicit Q-Learning for emotion selection and LoRA-based policy for emotion expression. EmoDistill-trained policies achieve the highest utility across four negotiation domains, outperforming vanilla baselines and IQL-only selection.
Post-trained LLMs are often optimized to align responses with human preferences, making them safe, polite, and conversationally appropriate. In adversarial negotiation, however, this alignment can become a vulnerability: emotionally framed language may steer agents toward the counterparty's interests. Using GoEmotions-based affective prompting, we show that emotion substantially shifts negotiation outcomes, suggesting that emotion is a strategic action channel rather than a surface style. Thus, we introduce \textbf{EmoDistill}, an offline framework for distilling emotional negotiation skills into language model agents. EmoDistill decomposes emotional strategy into emotion selection and emotion expression: an Implicit Q-Learning (IQL) selector learns \emph{which} emotion to express, while a Low-Rank Adaptation (LoRA)-based policy learns \emph{how} to express it through Supervised Fine-Tuning (SFT) and Judge Policy Optimization (JPO). Across four emotion-sensitive, high-stakes negotiation domains, SLM policies trained under the EmoDistill framework achieve the highest utility, outperforming vanilla SLM/LLM baselines and IQL-only emotion selection. Ablations show that emotion conditioning is essential, and transfer studies demonstrate generalization across domains, unseen counterparties, and trained-vs-trained tournaments. Overall, EmoDistill learns skills from offline agent-to-agent interactions, avoiding costly online negotiation during training.