Sequential Federated Learning in Hierarchical Architecture on Non-IID Datasets
This work addresses communication efficiency in federated learning systems, particularly for non-IID datasets, but is incremental as it combines existing hierarchical and sequential approaches.
The paper tackles the communication bottleneck in federated learning by introducing sequential federated learning into a hierarchical architecture, removing the central parameter server and enabling training through passing models between adjacent edge servers. The proposed Fed-CHS algorithm reduces communication overhead while maintaining comparable convergence performance and shows superiority in both overhead saving and test accuracy over baselines.
In a real federated learning (FL) system, communication overhead for passing model parameters between the clients and the parameter server (PS) is often a bottleneck. Hierarchical federated learning (HFL) that poses multiple edge servers (ESs) between clients and the PS can partially alleviate communication pressure but still needs the aggregation of model parameters from multiple ESs at the PS. To further reduce communication overhead, we bring sequential FL (SFL) into HFL for the first time, which removes the central PS and enables the model training to be completed only through passing the global model between two adjacent ESs for each iteration, and propose a novel algorithm adaptive to such a combinational framework, referred to as Fed-CHS. Convergence results are derived for strongly convex and non-convex loss functions under various data heterogeneity setups, which show comparable convergence performance with the algorithms for HFL or SFL solely. Experimental results provide evidence of the superiority of our proposed Fed-CHS on both communication overhead saving and test accuracy over baseline methods.