SoftAdaClip: A Smooth Clipping Strategy for Fair and Private Model Training
This addresses fairness and privacy issues in machine learning for sensitive domains like healthcare and income prediction, representing an incremental improvement over existing adaptive clipping methods.
The paper tackled the problem of differential privacy (DP) reducing model performance and fairness, especially for underrepresented groups, by introducing SoftAdaClip, a method that replaces hard gradient clipping with a smooth transformation. The results showed reductions in subgroup disparities of up to 87% compared to DP-SGD and up to 48% compared to Adaptive-DPSGD on datasets like MIMIC-III, GOSSIS-eICU, and Adult Income.
Differential privacy (DP) provides strong protection for sensitive data, but often reduces model performance and fairness, especially for underrepresented groups. One major reason is gradient clipping in DP-SGD, which can disproportionately suppress learning signals for minority subpopulations. Although adaptive clipping can enhance utility, it still relies on uniform hard clipping, which may restrict fairness. To address this, we introduce SoftAdaClip, a differentially private training method that replaces hard clipping with a smooth, tanh-based transformation to preserve relative gradient magnitudes while bounding sensitivity. We evaluate SoftAdaClip on various datasets, including MIMIC-III (clinical text), GOSSIS-eICU (structured healthcare), and Adult Income (tabular data). Our results show that SoftAdaClip reduces subgroup disparities by up to 87% compared to DP-SGD and up to 48% compared to Adaptive-DPSGD, and these reductions in subgroup disparities are statistically significant. These findings underscore the importance of integrating smooth transformations with adaptive mechanisms to achieve fair and private model training.