DP-REC: Private & Communication-Efficient Federated Learning
This addresses privacy and communication bottlenecks for federated learning systems, representing an incremental improvement by integrating existing techniques.
The paper tackled the combined challenges of privacy and communication efficiency in federated learning by developing DP-REC, a method that unifies differential privacy with compression, achieving drastically reduced communication costs while maintaining privacy comparable to state-of-the-art.
Privacy and communication efficiency are important challenges in federated training of neural networks, and combining them is still an open problem. In this work, we develop a method that unifies highly compressed communication and differential privacy (DP). We introduce a compression technique based on Relative Entropy Coding (REC) to the federated setting. With a minor modification to REC, we obtain a provably differentially private learning algorithm, DP-REC, and show how to compute its privacy guarantees. Our experiments demonstrate that DP-REC drastically reduces communication costs while providing privacy guarantees comparable to the state-of-the-art.