Aligning (Medical) LLMs for (Counterfactual) Fairness
This addresses fairness issues in medical AI tools to reduce health disparities, though it is incremental as it builds on existing bias mitigation techniques.
The study tackled biases in medical LLMs that cause unfair treatment and health disparities by proposing a new alignment approach using preference optimization within a knowledge distillation framework, showing it significantly reduces biased patterns in outputs.
Large Language Models (LLMs) have emerged as promising solutions for a variety of medical and clinical decision support applications. However, LLMs are often subject to different types of biases, which can lead to unfair treatment of individuals, worsening health disparities, and reducing trust in AI-augmented medical tools. Aiming to address this important issue, in this study, we present a new model alignment approach for aligning LLMs using a preference optimization method within a knowledge distillation framework. Prior to presenting our proposed method, we first use an evaluation framework to conduct a comprehensive (largest to our knowledge) empirical evaluation to reveal the type and nature of existing biases in LLMs used for medical applications. We then offer a bias mitigation technique to reduce the unfair patterns in LLM outputs across different subgroups identified by the protected attributes. We show that our mitigation method is effective in significantly reducing observed biased patterns. Our code is publicly available at \url{https://github.com/healthylaife/FairAlignmentLLM}.