Differentially Private Consensus-Based Distributed Optimization
This addresses privacy concerns in distributed learning for applications with sensitive data, but it is incremental as it relaxes previous differential privacy requirements.
The paper tackles the problem of data privacy in consensus-based distributed optimization by proposing a method where nodes add random noise to their messages, achieving convergence with bounded mean-squared error while satisfying (ε, δ)-differential privacy, and demonstrates effectiveness through numerical results for mean estimation.
Data privacy is an important concern in learning, when datasets contain sensitive information about individuals. This paper considers consensus-based distributed optimization under data privacy constraints. Consensus-based optimization consists of a set of computational nodes arranged in a graph, each having a local objective that depends on their local data, where in every step nodes take a linear combination of their neighbors' messages, as well as taking a new gradient step. Since the algorithm requires exchanging messages that depend on local data, private information gets leaked at every step. Taking $(ε, δ)$-differential privacy (DP) as our criterion, we consider the strategy where the nodes add random noise to their messages before broadcasting it, and show that the method achieves convergence with a bounded mean-squared error, while satisfying $(ε, δ)$-DP. By relaxing the more stringent $ε$-DP requirement in previous work, we strengthen a known convergence result in the literature. We conclude the paper with numerical results demonstrating the effectiveness of our methods for mean estimation.