Gender Bias in Large Language Models for Healthcare: Assignment Consistency and Clinical Implications
This addresses a critical bias issue for healthcare applications of LLMs, potentially undermining reliability in clinical practice, though it is incremental as it builds on known concerns about bias in AI.
The study investigated gender bias in large language models (LLMs) used in healthcare by assigning genders to LLMs and evaluating consistency in diagnoses and judgments on patient gender relevance, finding substantial inconsistency across LLM genders, particularly for relevance judgments, with some models showing systematic female-male disparities.
The integration of large language models (LLMs) into healthcare holds promise to enhance clinical decision-making, yet their susceptibility to biases remains a critical concern. Gender has long influenced physician behaviors and patient outcomes, raising concerns that LLMs assuming human-like roles, such as clinicians or medical educators, may replicate or amplify gender-related biases. Using case studies from the New England Journal of Medicine Challenge (NEJM), we assigned genders (female, male, or unspecified) to multiple open-source and proprietary LLMs. We evaluated their response consistency across LLM-gender assignments regarding both LLM-based diagnosis and models' judgments on the clinical relevance or necessity of patient gender. In our findings, diagnoses were relatively consistent across LLM genders for most models. However, for patient gender's relevance and necessity in LLM-based diagnosis, all models demonstrated substantial inconsistency across LLM genders, particularly for relevance judgements. Some models even displayed a systematic female-male disparity in their interpretation of patient gender. These findings present an underexplored bias that could undermine the reliability of LLMs in clinical practice, underscoring the need for routine checks of identity-assignment consistency when interacting with LLMs to ensure reliable and equitable AI-supported clinical care.