Measuring Normative and Descriptive Biases in Language Models Using Census Data
This work addresses the issue of bias in language models for researchers and practitioners in AI ethics, though it is incremental as it extends existing bias measurement methods to new data and languages.
The paper tackles the problem of measuring gender-occupation biases in pre-trained language models by comparing them to normative and descriptive distributions from census data across four countries, finding that models often reflect descriptive rather than normative realities.
We investigate in this paper how distributions of occupations with respect to gender is reflected in pre-trained language models. Such distributions are not always aligned to normative ideals, nor do they necessarily reflect a descriptive assessment of reality. In this paper, we introduce an approach for measuring to what degree pre-trained language models are aligned to normative and descriptive occupational distributions. To this end, we use official demographic information about gender--occupation distributions provided by the national statistics agencies of France, Norway, United Kingdom, and the United States. We manually generate template-based sentences combining gendered pronouns and nouns with occupations, and subsequently probe a selection of ten language models covering the English, French, and Norwegian languages. The scoring system we introduce in this work is language independent, and can be used on any combination of template-based sentences, occupations, and languages. The approach could also be extended to other dimensions of national census data and other demographic variables.