Hierarchical Federated Learning through LAN-WAN Orchestration
This addresses efficiency and cost problems for federated learning practitioners, though it appears incremental as it builds on existing FL frameworks with a novel network architecture.
The paper tackles the communication bottleneck and high costs in federated learning by proposing a hierarchical aggregation protocol that leverages local-area networks for frequent local updates and wide-area networks only for infrequent global aggregation. Their LanFL platform accelerates training by 1.5x-6.0x, reduces WAN traffic by 18.3x-75.6x, and cuts monetary costs by 3.8x-27.2x while maintaining model accuracy.
Federated learning (FL) was designed to enable mobile phones to collaboratively learn a global model without uploading their private data to a cloud server. However, exiting FL protocols has a critical communication bottleneck in a federated network coupled with privacy concerns, usually powered by a wide-area network (WAN). Such a WAN-driven FL design leads to significantly high cost and much slower model convergence. In this work, we propose an efficient FL protocol, which involves a hierarchical aggregation mechanism in the local-area network (LAN) due to its abundant bandwidth and almost negligible monetary cost than WAN. Our proposed FL can accelerate the learning process and reduce the monetary cost with frequent local aggregation in the same LAN and infrequent global aggregation on a cloud across WAN. We further design a concrete FL platform, namely LanFL, that incorporates several key techniques to handle those challenges introduced by LAN: cloud-device aggregation architecture, intra-LAN peer-to-peer (p2p) topology generation, inter-LAN bandwidth capacity heterogeneity. We evaluate LanFL on 2 typical Non-IID datasets, which reveals that LanFL can significantly accelerate FL training (1.5x-6.0x), save WAN traffic (18.3x-75.6x), and reduce monetary cost (3.8x-27.2x) while preserving the model accuracy.