Quantized Rank Reduction: A Communications-Efficient Federated Learning Scheme for Network-Critical Applications
This addresses communication efficiency for network-critical applications in federated learning, but it is incremental as it builds on existing methods.
The paper tackled the communication overhead problem in federated learning by proposing a scheme that uses low-rank approximation and quantization of gradients, resulting in significantly reduced network load with minimal accuracy impact.
Federated learning is a machine learning approach that enables multiple devices (i.e., agents) to train a shared model cooperatively without exchanging raw data. This technique keeps data localized on user devices, ensuring privacy and security, while each agent trains the model on their own data and only shares model updates. The communication overhead is a significant challenge due to the frequent exchange of model updates between the agents and the central server. In this paper, we propose a communication-efficient federated learning scheme that utilizes low-rank approximation of neural network gradients and quantization to significantly reduce the network load of the decentralized learning process with minimal impact on the model's accuracy.