On Mitigating the Utility-Loss in Differentially Private Learning: A new Perspective by a Geometrically Inspired Kernel Approach
This work addresses privacy concerns in machine learning for applications like biomedical data, though it appears incremental as it builds on existing kernel methods for differential privacy.
The paper tackles the privacy-utility tradeoff in differentially private machine learning by introducing a geometrically inspired kernel-based approach that learns representations in RKHS to hide sensitive information and reduce membership inference attacks, demonstrating effectiveness on datasets like MNIST with practical computational costs and marginal accuracy loss in federated learning.
Privacy-utility tradeoff remains as one of the fundamental issues of differentially private machine learning. This paper introduces a geometrically inspired kernel-based approach to mitigate the accuracy-loss issue in classification. In this approach, a representation of the affine hull of given data points is learned in Reproducing Kernel Hilbert Spaces (RKHS). This leads to a novel distance measure that hides privacy-sensitive information about individual data points and improves the privacy-utility tradeoff via significantly reducing the risk of membership inference attacks. The effectiveness of the approach is demonstrated through experiments on MNIST dataset, Freiburg groceries dataset, and a real biomedical dataset. It is verified that the approach remains computationally practical. The application of the approach to federated learning is considered and it is observed that the accuracy-loss due to data being distributed is either marginal or not significantly high.