Mitigating Data Heterogeneity in Federated Learning with Data Augmentation
This addresses the problem of non-IID data in federated learning for privacy-preserving machine learning, but it is incremental as it adapts existing domain generalization methods.
The paper tackles data heterogeneity in federated learning by applying data augmentation, showing that this approach mitigates the issue and achieves state-of-the-art performance with higher accuracy on unseen clients and sparser communication.
Federated Learning (FL) is a prominent framework that enables training a centralized model while securing user privacy by fusing local, decentralized models. In this setting, one major obstacle is data heterogeneity, i.e., each client having non-identically and independently distributed (non-IID) data. This is analogous to the context of Domain Generalization (DG), where each client can be treated as a different domain. However, while many approaches in DG tackle data heterogeneity from the algorithmic perspective, recent evidence suggests that data augmentation can induce equal or greater performance. Motivated by this connection, we present federated versions of popular DG algorithms, and show that by applying appropriate data augmentation, we can mitigate data heterogeneity in the federated setting, and obtain higher accuracy on unseen clients. Equipped with data augmentation, we can achieve state-of-the-art performance using even the most basic Federated Averaging algorithm, with much sparser communication.