Capture Global Feature Statistics for One-Shot Federated Learning
This addresses communication efficiency and data heterogeneity issues in federated learning for distributed systems, representing an incremental improvement over existing one-shot methods.
The paper tackles the problem of high communication costs and privacy risks in federated learning by proposing FedCGS, a one-shot method that captures global feature statistics using pre-trained models, achieving training-free and heterogeneity-resistant performance with stable results across non-IID data settings.
Traditional Federated Learning (FL) necessitates numerous rounds of communication between the server and clients, posing significant challenges including high communication costs, connection drop risks and susceptibility to privacy attacks. One-shot FL has become a compelling learning paradigm to overcome above drawbacks by enabling the training of a global server model via a single communication round. However, existing one-shot FL methods suffer from expensive computation cost on the server or clients and cannot deal with non-IID (Independent and Identically Distributed) data stably and effectively. To address these challenges, this paper proposes FedCGS, a novel Federated learning algorithm that Capture Global feature Statistics leveraging pre-trained models. With global feature statistics, we achieve training-free and heterogeneity-resistant one-shot FL. Furthermore, we extend its application to personalization scenario, where clients only need execute one extra communication round with server to download global statistics. Extensive experimental results demonstrate the effectiveness of our methods across diverse data heterogeneity settings. Code is available at https://github.com/Yuqin-G/FedCGS.