One Size Fits None: Rethinking Fairness in Medical AI
This work addresses fairness issues in medical AI for clinicians and developers, but it is incremental as it focuses on analysis and discussion rather than introducing new methods.
The paper tackles the problem of performance disparities in machine learning models for medical prediction tasks across patient subgroups, arguing that subgroup-level evaluation is essential before clinical integration to identify and address fairness concerns.
Machine learning (ML) models are increasingly used to support clinical decision-making. However, real-world medical datasets are often noisy, incomplete, and imbalanced, leading to performance disparities across patient subgroups. These differences raise fairness concerns, particularly when they reinforce existing disadvantages for marginalized groups. In this work, we analyze several medical prediction tasks and demonstrate how model performance varies with patient characteristics. While ML models may demonstrate good overall performance, we argue that subgroup-level evaluation is essential before integrating them into clinical workflows. By conducting a performance analysis at the subgroup level, differences can be clearly identified-allowing, on the one hand, for performance disparities to be considered in clinical practice, and on the other hand, for these insights to inform the responsible development of more effective models. Thereby, our work contributes to a practical discussion around the subgroup-sensitive development and deployment of medical ML models and the interconnectedness of fairness and transparency.