Addressing Heterogeneity in Federated Learning via Distributional Transformation
This addresses the problem of poor model performance in federated learning for clients with heterogeneous data distributions, representing a novel method for a known bottleneck.
The paper tackles the challenge of data heterogeneity in federated learning by proposing DisTrans, a framework that uses distributional transformations and a double-input-channel model to improve model accuracy, achieving superior performance over state-of-the-art methods across multiple benchmark datasets under varying heterogeneity levels.
Federated learning (FL) allows multiple clients to collaboratively train a deep learning model. One major challenge of FL is when data distribution is heterogeneous, i.e., differs from one client to another. Existing personalized FL algorithms are only applicable to narrow cases, e.g., one or two data classes per client, and therefore they do not satisfactorily address FL under varying levels of data heterogeneity. In this paper, we propose a novel framework, called DisTrans, to improve FL performance (i.e., model accuracy) via train and test-time distributional transformations along with a double-input-channel model structure. DisTrans works by optimizing distributional offsets and models for each FL client to shift their data distribution, and aggregates these offsets at the FL server to further improve performance in case of distributional heterogeneity. Our evaluation on multiple benchmark datasets shows that DisTrans outperforms state-of-the-art FL methods and data augmentation methods under various settings and different degrees of client distributional heterogeneity.