Communication-Efficient Federated Learning via Predictive Coding
This addresses the critical problem of high communication overhead for resource-constrained devices in federated learning, representing a strong incremental improvement over existing compression methods.
The paper tackles the communication bottleneck in federated learning for wireless mobile devices by proposing a predictive coding-based compression scheme, achieving up to 99% reduction in communication cost with improved learning performance compared to baselines.
Federated learning can enable remote workers to collaboratively train a shared machine learning model while allowing training data to be kept locally. In the use case of wireless mobile devices, the communication overhead is a critical bottleneck due to limited power and bandwidth. Prior work has utilized various data compression tools such as quantization and sparsification to reduce the overhead. In this paper, we propose a predictive coding based compression scheme for federated learning. The scheme has shared prediction functions among all devices and allows each worker to transmit a compressed residual vector derived from the reference. In each communication round, we select the predictor and quantizer based on the rate-distortion cost, and further reduce the redundancy with entropy coding. Extensive simulations reveal that the communication cost can be reduced up to 99% with even better learning performance when compared with other baseline methods.