Federated Learning Beyond the Star: Local D2D Model Consensus with Global Cluster Sampling
This work addresses scalability and efficiency issues in federated learning for edge computing applications, representing an incremental advancement over existing methods.
The paper tackles the problem of federated learning's reliance on a central server by proposing a two-timescale hybrid approach that uses device-to-device communications for local consensus and global cluster sampling, achieving a convergence rate of O(1/t) and showing improvements in convergence and utilization over baselines.
Federated learning has emerged as a popular technique for distributing model training across the network edge. Its learning architecture is conventionally a star topology between the devices and a central server. In this paper, we propose two timescale hybrid federated learning (TT-HF), which migrates to a more distributed topology via device-to-device (D2D) communications. In TT-HF, local model training occurs at devices via successive gradient iterations, and the synchronization process occurs at two timescales: (i) macro-scale, where global aggregations are carried out via device-server interactions, and (ii) micro-scale, where local aggregations are carried out via D2D cooperative consensus formation in different device clusters. Our theoretical analysis reveals how device, cluster, and network-level parameters affect the convergence of TT-HF, and leads to a set of conditions under which a convergence rate of O(1/t) is guaranteed. Experimental results demonstrate the improvements in convergence and utilization that can be obtained by TT-HF over state-of-the-art federated learning baselines.