Mitigating Strategy Preference Bias in Emotional Support Conversation via Uncertainty Estimations
This work addresses a specific bias issue in emotional support dialogue systems, offering an incremental improvement for enhancing the accuracy and fairness of strategy planning in LLMs.
The paper tackled the problem of strategy preference bias in emotional support conversation (ESC) by LLMs, revealing its causes through knowledge boundaries and proposing a reinforcement learning approach with a dual reward function to mitigate it, achieving improved performance on ESCov and ExTES datasets.
Emotional support conversation (ESC) aims to alleviate distress through empathetic dialogue, yet large language models (LLMs) face persistent challenges in delivering effective ESC due to low accuracy in strategy planning. Moreover, there is a considerable preference bias towards specific strategies. Prior methods using fine-tuned strategy planners have shown potential in reducing such bias, while the underlying causes of the preference bias in LLMs have not well been studied. To address these issues, we first reveal the fundamental causes of the bias by identifying the knowledge boundaries of LLMs in strategy planning. Then, we propose an approach to mitigate the bias by reinforcement learning with a dual reward function, which optimizes strategy planning via both accuracy and entropy-based confidence for each region according to the knowledge boundaries. Experiments on the ESCov and ExTES datasets with multiple LLM backbones show that our approach outperforms the baselines, confirming the effectiveness of our approach.