The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches
This work addresses privacy-preserving data analysis for applications like federated learning, but it is incremental as it refines existing methods rather than introducing a new paradigm.
The paper revisits Gaussian sketching for differential privacy, providing a refined Renyi Differential Privacy analysis that yields tighter bounds than prior results, and demonstrates performance improvements in linear regression settings with empirical gains on multiple datasets and reduced runtime in some cases.
Gaussian sketching, which consists of pre-multiplying the data with a random Gaussian matrix, is a widely used technique for multiple problems in data science and machine learning, with applications spanning computationally efficient optimization, coded computing, and federated learning. This operation also provides differential privacy guarantees due to its inherent randomness. In this work, we revisit this operation through the lens of Renyi Differential Privacy (RDP), providing a refined privacy analysis that yields significantly tighter bounds than prior results. We then demonstrate how this improved analysis leads to performance improvement in different linear regression settings, establishing theoretical utility guarantees. Empirically, our methods improve performance across multiple datasets and, in several cases, reduce runtime.