Detecting Natural Language Biases with Prompt-based Learning
This addresses bias detection in language models for AI fairness, but it is incremental as it applies prompt-based learning to an existing task.
The study tackled detecting biases in language models by designing prompts for gender, race, sexual orientation, and religion-based biases, applying them to BERT, RoBERTa, and T5, and evaluating with human and model-level judgments.
In this project, we want to explore the newly emerging field of prompt engineering and apply it to the downstream task of detecting LM biases. More concretely, we explore how to design prompts that can indicate 4 different types of biases: (1) gender, (2) race, (3) sexual orientation, and (4) religion-based. Within our project, we experiment with different manually crafted prompts that can draw out the subtle biases that may be present in the language model. We apply these prompts to multiple variations of popular and well-recognized models: BERT, RoBERTa, and T5 to evaluate their biases. We provide a comparative analysis of these models and assess them using a two-fold method: use human judgment to decide whether model predictions are biased and utilize model-level judgment (through further prompts) to understand if a model can self-diagnose the biases of its own prediction.