Fair Distillation: Teaching Fairness from Biased Teachers in Medical Imaging
This addresses fairness issues in medical imaging for demographic groups, but it is incremental as it builds on existing bias-mitigation techniques.
The paper tackled the problem of fairness in deep learning models for medical imaging, where biases affect demographic groups, by proposing the Fair Distillation (FairDi) method, which uses biased teacher models to train a student model, achieving significant gains in overall and group-specific accuracy and improved fairness compared to existing methods.
Deep learning has achieved remarkable success in image classification and segmentation tasks. However, fairness concerns persist, as models often exhibit biases that disproportionately affect demographic groups defined by sensitive attributes such as race, gender, or age. Existing bias-mitigation techniques, including Subgroup Re-balancing, Adversarial Training, and Domain Generalization, aim to balance accuracy across demographic groups, but often fail to simultaneously improve overall accuracy, group-specific accuracy, and fairness due to conflicts among these interdependent objectives. We propose the Fair Distillation (FairDi) method, a novel fairness approach that decomposes these objectives by leveraging biased ``teacher'' models, each optimized for a specific demographic group. These teacher models then guide the training of a unified ``student'' model, which distills their knowledge to maximize overall and group-specific accuracies, while minimizing inter-group disparities. Experiments on medical imaging datasets show that FairDi achieves significant gains in both overall and group-specific accuracy, along with improved fairness, compared to existing methods. FairDi is adaptable to various medical tasks, such as classification and segmentation, and provides an effective solution for equitable model performance.