NLPositionality: Characterizing Design Biases of Datasets and Models
This addresses the issue of inclusivity and fairness in NLP for researchers and developers, though it is incremental as it builds on prior work to provide a new quantification method.
The authors tackled the problem of design biases in NLP systems by introducing NLPositionality, a framework to quantify positionality in datasets and models, finding that they predominantly align with Western, White, college-educated, and younger populations, with certain groups like non-binary people and non-native English speakers being marginalized.
Design biases in NLP systems, such as performance differences for different populations, often stem from their creator's positionality, i.e., views and lived experiences shaped by identity and background. Despite the prevalence and risks of design biases, they are hard to quantify because researcher, system, and dataset positionality is often unobserved. We introduce NLPositionality, a framework for characterizing design biases and quantifying the positionality of NLP datasets and models. Our framework continuously collects annotations from a diverse pool of volunteer participants on LabintheWild, and statistically quantifies alignment with dataset labels and model predictions. We apply NLPositionality to existing datasets and models for two tasks -- social acceptability and hate speech detection. To date, we have collected 16,299 annotations in over a year for 600 instances from 1,096 annotators across 87 countries. We find that datasets and models align predominantly with Western, White, college-educated, and younger populations. Additionally, certain groups, such as non-binary people and non-native English speakers, are further marginalized by datasets and models as they rank least in alignment across all tasks. Finally, we draw from prior literature to discuss how researchers can examine their own positionality and that of their datasets and models, opening the door for more inclusive NLP systems.