Enforcing Privacy in Distributed Learning with Performance Guarantees
This work addresses privacy concerns in distributed learning for applications like federated learning, offering a more practical solution by removing unrealistic assumptions, though it is incremental in improving existing methods.
The paper tackles the problem of performance degradation in differentially private distributed learning by proposing graph-homomorphic constructions that improve performance while ensuring privacy, without requiring bounded gradients as in prior work, and demonstrates these findings through simulations.
We study the privatization of distributed learning and optimization strategies. We focus on differential privacy schemes and study their effect on performance. We show that the popular additive random perturbation scheme degrades performance because it is not well-tuned to the graph structure. For this reason, we exploit two alternative graph-homomorphic constructions and show that they improve performance while guaranteeing privacy. Moreover, contrary to most earlier studies, the gradient of the risks is not assumed to be bounded (a condition that rarely holds in practice; e.g., quadratic risk). We avoid this condition and still devise a differentially private scheme with high probability. We examine optimization and learning scenarios and illustrate the theoretical findings through simulations.