Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models
This work addresses the issue of harmful stereotype reproduction in NLP models for marginalized groups, though it is incremental as it builds on existing psychological frameworks.
The paper tackled the problem of measuring social stereotypes in language models by adapting the Agency-Belief-Communion framework from social psychology and introducing the sensitivity test (SeT) for systematic evaluation, finding that it enables comparison with human judgments and extension to intersectional identities.
NLP models trained on text have been shown to reproduce human stereotypes, which can magnify harms to marginalized groups when systems are deployed at scale. We adapt the Agency-Belief-Communion (ABC) stereotype model of Koch et al. (2016) from social psychology as a framework for the systematic study and discovery of stereotypic group-trait associations in language models (LMs). We introduce the sensitivity test (SeT) for measuring stereotypical associations from language models. To evaluate SeT and other measures using the ABC model, we collect group-trait judgments from U.S.-based subjects to compare with English LM stereotypes. Finally, we extend this framework to measure LM stereotyping of intersectional identities.