Synthetic Data Aided Federated Learning Using Foundation Models
This addresses the problem of data heterogeneity in FL for distributed machine learning applications, representing an incremental improvement with a novel data augmentation strategy.
The paper tackles performance degradation in Federated Learning (FL) due to non-IID data heterogeneity by proposing DPSDA-FL, a method using differentially private synthetic data from foundation models to homogenize local data, resulting in up to 26% improvement in class recall and 9% in classification accuracy on CIFAR-10.
In heterogeneous scenarios where the data distribution amongst the Federated Learning (FL) participants is Non-Independent and Identically distributed (Non-IID), FL suffers from the well known problem of data heterogeneity. This leads the performance of FL to be significantly degraded, as the global model tends to struggle to converge. To solve this problem, we propose Differentially Private Synthetic Data Aided Federated Learning Using Foundation Models (DPSDA-FL), a novel data augmentation strategy that aids in homogenizing the local data present on the clients' side. DPSDA-FL improves the training of the local models by leveraging differentially private synthetic data generated from foundation models. We demonstrate the effectiveness of our approach by evaluating it on the benchmark image dataset: CIFAR-10. Our experimental results have shown that DPSDA-FL can improve class recall and classification accuracy of the global model by up to 26% and 9%, respectively, in FL with Non-IID issues.