Federated Generative Learning with Foundation Models
This addresses communication and privacy problems in federated learning for distributed AI applications, representing a novel method rather than an incremental improvement.
The paper tackles inefficiency, privacy, and security issues in Federated Learning by proposing Federated Generative Learning, where clients send text embeddings to a server that synthesizes training data using foundation models, resulting in a 12% performance improvement over FedAvg on ImageNet100 with skewed data.
Existing approaches in Federated Learning (FL) mainly focus on sending model parameters or gradients from clients to a server. However, these methods are plagued by significant inefficiency, privacy, and security concerns. Thanks to the emerging foundation generative models, we propose a novel federated learning framework, namely Federated Generative Learning. In this framework, each client can create text embeddings that are tailored to their local data, and send embeddings to the server. Then the informative training data can be synthesized remotely on the server using foundation generative models with these embeddings, which can benefit FL tasks. Our proposed framework offers several advantages, including increased communication efficiency, robustness to data heterogeneity, substantial performance improvements, and enhanced privacy protection. We validate these benefits through extensive experiments conducted on 12 datasets. For example, on the ImageNet100 dataset with a highly skewed data distribution, our method outperforms FedAvg by 12% in a single communication round, compared to FedAvg's performance over 200 communication rounds. We have released the code for all experiments conducted in this study.