APPLE: Adversarial Privacy-aware Perturbations on Latent Embedding for Unfairness Mitigation
This addresses fairness issues in medical image segmentation for health equity, offering a post-hoc solution that is incremental by building on existing models without weight updates.
The paper tackles unfairness in deployed deep-learning-based medical image segmentors by proposing APPLE, a method that adds adversarial perturbations to latent embeddings to decorrelate fairness-related features without retraining, achieving improved fairness across multiple segmentors and datasets.
Ensuring fairness in deep-learning-based segmentors is crucial for health equity. Much effort has been dedicated to mitigating unfairness in the training datasets or procedures. However, with the increasing prevalence of foundation models in medical image analysis, it is hard to train fair models from scratch while preserving utility. In this paper, we propose a novel method, Adversarial Privacy-aware Perturbations on Latent Embedding (APPLE), that can improve the fairness of deployed segmentors by introducing a small latent feature perturber without updating the weights of the original model. By adding perturbation to the latent vector, APPLE decorates the latent vector of segmentors such that no fairness-related features can be passed to the decoder of the segmentors while preserving the architecture and parameters of the segmentor. Experiments on two segmentation datasets and five segmentors (three U-Net-like and two SAM-like) illustrate the effectiveness of our proposed method compared to several unfairness mitigation methods.