Differentially Private Truncation of Unbounded Data via Public Second Moments
This addresses a key bottleneck in applying DP to unbounded data for privacy-sensitive AI applications, though it is an incremental improvement building on existing DP frameworks.
The paper tackles the limitation of differential privacy (DP) requiring bounded data distributions by proposing Public-moment-guided Truncation (PMT), which uses public second-moment information to transform and truncate private data, resulting in improved accuracy and stability of DP models as confirmed by experiments.
Data privacy is important in the AI era, and differential privacy (DP) is one of the golden solutions. However, DP is typically applicable only if data have a bounded underlying distribution. We address this limitation by leveraging second-moment information from a small amount of public data. We propose Public-moment-guided Truncation (PMT), which transforms private data using the public second-moment matrix and applies a principled truncation whose radius depends only on non-private quantities: data dimension and sample size. This transformation yields a well-conditioned second-moment matrix, enabling its inversion with a significantly strengthened ability to resist the DP noise. Furthermore, we demonstrate the applicability of PMT by using penalized and generalized linear regressions. Specifically, we design new loss functions and algorithms, ensuring that solutions in the transformed space can be mapped back to the original domain. We have established improvements in the models' DP estimation through theoretical error bounds, robustness guarantees, and convergence results, attributing the gains to the conditioning effect of PMT. Experiments on synthetic and real datasets confirm that PMT substantially improves the accuracy and stability of DP models.