Killing Two Birds with One Stone: Quantization Achieves Privacy in Distributed Learning
This work addresses privacy and efficiency issues for distributed learning systems, particularly in resource-limited environments, though it is incremental as it builds on existing SGD frameworks.
The paper tackles the dual challenges of communication efficiency and privacy protection in distributed machine learning by proposing a quantization-based solution that adds binomial noise to quantized gradients, achieving differential privacy with minimal communication overhead.
Communication efficiency and privacy protection are two critical issues in distributed machine learning. Existing methods tackle these two issues separately and may have a high implementation complexity that constrains their application in a resource-limited environment. We propose a comprehensive quantization-based solution that could simultaneously achieve communication efficiency and privacy protection, providing new insights into the correlated nature of communication and privacy. Specifically, we demonstrate the effectiveness of our proposed solutions in the distributed stochastic gradient descent (SGD) framework by adding binomial noise to the uniformly quantized gradients to reach the desired differential privacy level but with a minor sacrifice in communication efficiency. We theoretically capture the new trade-offs between communication, privacy, and learning performance.