Wireless Federated Distillation for Distributed Edge Learning with Heterogeneous Data
This work addresses the challenge of enabling efficient distributed machine learning in real-world wireless environments with heterogeneous data, which is incremental as it builds on existing FL and FD methods.
The paper tackles the problem of distributed edge learning over noisy wireless channels by proposing and evaluating digital and over-the-air implementations of Federated Learning, Federated Distillation, and a novel Hybrid Federated Distillation scheme over Gaussian multiple-access channels.
Cooperative training methods for distributed machine learning typically assume noiseless and ideal communication channels. This work studies some of the opportunities and challenges arising from the presence of wireless communication links. We specifically consider wireless implementations of Federated Learning (FL) and Federated Distillation (FD), as well as of a novel Hybrid Federated Distillation (HFD) scheme. Both digital implementations based on separate source-channel coding and over-the-air computing implementations based on joint source-channel coding are proposed and evaluated over Gaussian multiple-access channels.