The Effect of Enforcing Fairness on Reshaping Explanations in Machine Learning Models
This addresses the problem of ensuring trustworthy AI in healthcare by highlighting the trade-offs between fairness and explainability, which is incremental as it builds on known fairness-performance interactions.
The study examined how applying fairness constraints to machine learning models reshapes Shapley-based feature importance rankings, finding that increasing fairness across racial subgroups can significantly alter these rankings, sometimes differently across groups.
Trustworthy machine learning in healthcare requires strong predictive performance, fairness, and explanations. While it is known that improving fairness can affect predictive performance, little is known about how fairness improvements influence explainability, an essential ingredient for clinical trust. Clinicians may hesitate to rely on a model whose explanations shift after fairness constraints are applied. In this study, we examine how enhancing fairness through bias mitigation techniques reshapes Shapley-based feature rankings. We quantify changes in feature importance rankings after applying fairness constraints across three datasets: pediatric urinary tract infection risk, direct anticoagulant bleeding risk, and recidivism risk. We also evaluate multiple model classes on the stability of Shapley-based rankings. We find that increasing model fairness across racial subgroups can significantly alter feature importance rankings, sometimes in different ways across groups. These results highlight the need to jointly consider accuracy, fairness, and explainability in model assessment rather than in isolation.