Modeling Global Distribution for Federated Learning with Label Distribution Skew
This addresses performance degradation in federated learning for scenarios with non-IID label distributions, offering a privacy-preserving solution, though it is an incremental improvement over existing methods.
The paper tackles the problem of label distribution skew in federated learning, which degrades global model performance, by proposing FedMGD, a method that uses a global Generative Adversarial Network to model data distribution without accessing local datasets, and it significantly outperforms state-of-the-art methods on public benchmarks.
Federated learning achieves joint training of deep models by connecting decentralized data sources, which can significantly mitigate the risk of privacy leakage. However, in a more general case, the distributions of labels among clients are different, called ``label distribution skew''. Directly applying conventional federated learning without consideration of label distribution skew issue significantly hurts the performance of the global model. To this end, we propose a novel federated learning method, named FedMGD, to alleviate the performance degradation caused by the label distribution skew issue. It introduces a global Generative Adversarial Network to model the global data distribution without access to local datasets, so the global model can be trained using the global information of data distribution without privacy leakage. The experimental results demonstrate that our proposed method significantly outperforms the state-of-the-art on several public benchmarks. Code is available at \url{https://github.com/Sheng-T/FedMGD}.