FedHeN: Federated Learning in Heterogeneous Networks
This addresses the problem of efficient federated learning in heterogeneous networks for applications like edge computing, though it appears incremental.
The paper tackles federated learning with heterogeneous device architectures by introducing a side objective for more complex devices, resulting in improved performance across architectures and significant communication savings compared to state-of-the-art methods.
We propose a novel training recipe for federated learning with heterogeneous networks where each device can have different architectures. We introduce training with a side objective to the devices of higher complexities to jointly train different architectures in a federated setting. We empirically show that our approach improves the performance of different architectures and leads to high communication savings compared to the state-of-the-art methods.