COMPEER: Controllable Empathetic Reinforcement Reasoning for Emotional Support Conversation
This work addresses the need for more empathetic and human-like support systems in mental health applications, representing an incremental advancement in the field.
The paper tackles the problem of enabling deep empathetic reasoning in emotional support conversations by proposing a controllable reasoning approach that integrates natural language reasoning with structured psychological steps, resulting in significant improvement in the model's emotional support ability.
Emotional support conversations are crucial for promoting emotional well-being, yet current models often lack deep empathetic reasoning grounded in psychological principles. To address this, we propose controllable empathetic reasoning, which combines natural language reasoning with structured psychological steps. We construct a fine-grained dataset annotated with reasoning correctness and response preferences to enable this capability. To further enhance training, we employ reinforcement learning with a unified process-outcome reward model that delivers precise feedback. To mitigate response repetitiveness from entropy collapse, we introduce personality-based dialogue rewriting and a redundancy-aware reward reweighting strategy. Our approach significantly improves model's emotional support ability, advancing the development of empathetic, human-like support systems.