Are LLMs Empathetic to All? Investigating the Influence of Multi-Demographic Personas on a Model's Empathy
This addresses the problem of biased or inequitable empathy in LLMs for users from diverse demographic backgrounds, which is incremental as it builds on existing empathy research by adding intersectional analysis.
The study investigated whether large language models (LLMs) demonstrate equitable empathy across diverse user groups by analyzing cognitive and affective empathy across 315 personas defined by age, culture, and gender, finding that attributes profoundly shape empathetic responses with notable misalignments for groups like Confucian culture.
Large Language Models' (LLMs) ability to converse naturally is empowered by their ability to empathetically understand and respond to their users. However, emotional experiences are shaped by demographic and cultural contexts. This raises an important question: Can LLMs demonstrate equitable empathy across diverse user groups? We propose a framework to investigate how LLMs' cognitive and affective empathy vary across user personas defined by intersecting demographic attributes. Our study introduces a novel intersectional analysis spanning 315 unique personas, constructed from combinations of age, culture, and gender, across four LLMs. Results show that attributes profoundly shape a model's empathetic responses. Interestingly, we see that adding multiple attributes at once can attenuate and reverse expected empathy patterns. We show that they broadly reflect real-world empathetic trends, with notable misalignments for certain groups, such as those from Confucian culture. We complement our quantitative findings with qualitative insights to uncover model behaviour patterns across different demographic groups. Our findings highlight the importance of designing empathy-aware LLMs that account for demographic diversity to promote more inclusive and equitable model behaviour.