MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcare
This addresses the challenge of data heterogeneity and distrust in federated learning for personalized healthcare, offering a serverless solution that is incremental over existing FL methods.
The paper tackles the problem of enabling federated learning across different federations without a central server, particularly in healthcare, by proposing MetaFed with Cyclic Knowledge Distillation, achieving over 10% accuracy improvement on benchmarks like PAMAP2 with reduced communication costs.
Federated learning has attracted increasing attention to building models without accessing the raw user data, especially in healthcare. In real applications, different federations can seldom work together due to possible reasons such as data heterogeneity and distrust/inexistence of the central server. In this paper, we propose a novel framework called MetaFed to facilitate trustworthy FL between different federations. MetaFed obtains a personalized model for each federation without a central server via the proposed Cyclic Knowledge Distillation. Specifically, MetaFed treats each federation as a meta distribution and aggregates knowledge of each federation in a cyclic manner. The training is split into two parts: common knowledge accumulation and personalization. Comprehensive experiments on three benchmarks demonstrate that MetaFed without a server achieves better accuracy compared to state-of-the-art methods (e.g., 10%+ accuracy improvement compared to the baseline for PAMAP2) with fewer communication costs.