Integrating Feature Correlation in Differential Privacy with Applications in DP-ERM
For practitioners needing differential privacy with heterogeneous feature sensitivity, this work provides a principled relaxation that improves utility while maintaining privacy guarantees.
The paper introduces CorrDP, a relaxed differential privacy definition that accounts for heterogeneous feature sensitivity by allowing insensitive features to be treated as such even when correlated with sensitive ones. They design DP-ERM algorithms under CorrDP that achieve improved utility, outperforming standard DP in experiments.
Standard differential privacy imposes uniform privacy constraints across all features, overlooking the inherent distinction between sensitive and insensitive features in practice. In this paper, we introduce a relaxed definition of differential privacy that accounts for such heterogeneity, allowing certain features to be treated as insensitive even when correlated with sensitive ones. We propose a correlation-aware framework, $\textsf{CorrDP}$, which relaxes privacy for insensitive features while accounting for their correlations with sensitive features, with the correlations quantified using total variation distance. We design algorithms for differentially private empirical risk minimization (DP-ERM) under the $\textsf{CorrDP}$ framework, incorporating distance-dependent noise into gradients for improved theoretical utility guarantees. When the correlation distance is unknown, we estimate it from the dataset and show that it achieves a comparable privacy-utility guarantee. We perform experiments on synthetic and real-world datasets and show that $\textsf{CorrDP}$-based DP-ERM algorithms consistently outperform the standard DP framework in the presence of insensitive features.