Training Federated GANs with Theoretical Guarantees: A Universal Aggregation Approach
This work provides a provably correct framework for federated GANs, which is significant for researchers and practitioners working on privacy-preserving machine learning, particularly when dealing with heterogeneous data distributions across different sites. It addresses a fundamental theoretical limitation of existing federated GAN approaches.
This paper addresses the theoretical challenge of data heterogeneity in federated Generative Adversarial Networks (GANs), where traditional methods fail to learn the target distribution, a mixture of diverse local distributions. The authors propose a Universal Aggregation approach that simulates a centralized discriminator by aggregating private discriminators, proving that a generator trained with this simulated discriminator can learn the desired target distribution. Experiments on synthetic and real datasets demonstrate its ability to learn mixtures of largely different distributions where existing federated GANs fail.
Recently, Generative Adversarial Networks (GANs) have demonstrated their potential in federated learning, i.e., learning a centralized model from data privately hosted by multiple sites. A federatedGAN jointly trains a centralized generator and multiple private discriminators hosted at different sites. A major theoretical challenge for the federated GAN is the heterogeneity of the local data distributions. Traditional approaches cannot guarantee to learn the target distribution, which isa mixture of the highly different local distributions. This paper tackles this theoretical challenge, and for the first time, provides a provably correct framework for federated GAN. We propose a new approach called Universal Aggregation, which simulates a centralized discriminator via carefully aggregating the mixture of all private discriminators. We prove that a generator trained with this simulated centralized discriminator can learn the desired target distribution. Through synthetic and real datasets, we show that our method can learn the mixture of largely different distributions where existing federated GAN methods fail.