Assessing GPT's Bias Towards Stigmatized Social Groups: An Intersectional Case Study on Nationality Prejudice and Psychophobia
This research addresses ethical concerns in AI for marginalized communities, highlighting incremental findings on intersectional biases in widely-used LLMs.
The study investigated biases in GPT models against stigmatized social groups, focusing on nationality and mental disability intersections, and found significant negative biases, especially against North Koreans with mental disabilities, with empathy levels showing notable discrepancies.
Recent studies have separately highlighted significant biases within foundational large language models (LLMs) against certain nationalities and stigmatized social groups. This research investigates the ethical implications of these biases intersecting with outputs of widely-used GPT-3.5/4/4o LLMS. Through structured prompt series, we evaluate model responses to several scenarios involving American and North Korean nationalities with various mental disabilities. Findings reveal significant discrepancies in empathy levels with North Koreans facing greater negative bias, particularly when mental disability is also a factor. This underscores the need for improvements in LLMs designed with a nuanced understanding of intersectional identity.