Communication Efficient and Privacy-Preserving Federated Learning Based on Evolution Strategies
This addresses communication efficiency and privacy issues in distributed deep learning for applications like edge computing, though it is incremental as it adapts an existing zeroth-order method to federated learning.
The paper tackled the high communication overhead and privacy risks in federated learning by proposing FedES, a method based on evolution strategies that communicates only loss values, achieving low communication costs and privacy protection while maintaining convergence performance comparable to backpropagation methods.
Federated learning (FL) is an emerging paradigm for training deep neural networks (DNNs) in distributed manners. Current FL approaches all suffer from high communication overhead and information leakage. In this work, we present a federated learning algorithm based on evolution strategies (FedES), a zeroth-order training method. Instead of transmitting model parameters, FedES only communicates loss values, and thus has very low communication overhead. Moreover, a third party is unable to estimate gradients without knowing the pre-shared seed, which protects data privacy. Experimental results demonstrate FedES can achieve the above benefits while keeping convergence performance the same as that with back propagation methods.