Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
This work addresses fairness issues in clinical AI systems, which can perpetuate biases against marginalized populations, though it is incremental as it builds on existing bias quantification methods.
The study quantified biases in clinical contextual word embeddings trained on MIMIC-III medical notes, finding that classifiers using these embeddings showed statistically significant performance gaps favoring majority groups in over 50 clinical prediction tasks.
In this work, we examine the extent to which embeddings may encode marginalized populations differently, and how this may lead to a perpetuation of biases and worsened performance on clinical tasks. We pretrain deep embedding models (BERT) on medical notes from the MIMIC-III hospital dataset, and quantify potential disparities using two approaches. First, we identify dangerous latent relationships that are captured by the contextual word embeddings using a fill-in-the-blank method with text from real clinical notes and a log probability bias score quantification. Second, we evaluate performance gaps across different definitions of fairness on over 50 downstream clinical prediction tasks that include detection of acute and chronic conditions. We find that classifiers trained from BERT representations exhibit statistically significant differences in performance, often favoring the majority group with regards to gender, language, ethnicity, and insurance status. Finally, we explore shortcomings of using adversarial debiasing to obfuscate subgroup information in contextual word embeddings, and recommend best practices for such deep embedding models in clinical settings.