A New Distributed Method for Training Generative Adversarial Networks
This addresses the challenge of distributed GAN training for applications with decentralized data, though it appears incremental as it builds on existing distributed methods.
The paper tackles the problem of training generative adversarial networks (GANs) in distributed settings with privacy and communication constraints, proposing a new framework where devices compute local discriminators and a server aggregates them into a global GAN. Numerical results show it outperforms a state-of-the-art framework in convergence speed on three datasets.
Generative adversarial networks (GANs) are emerging machine learning models for generating synthesized data similar to real data by jointly training a generator and a discriminator. In many applications, data and computational resources are distributed over many devices, so centralized computation with all data in one location is infeasible due to privacy and/or communication constraints. This paper proposes a new framework for training GANs in a distributed fashion: Each device computes a local discriminator using local data; a single server aggregates their results and computes a global GAN. Specifically, in each iteration, the server sends the global GAN to the devices, which then update their local discriminators; the devices send their results to the server, which then computes their average as the global discriminator and updates the global generator accordingly. Two different update schedules are designed with different levels of parallelism between the devices and the server. Numerical results obtained using three popular datasets demonstrate that the proposed framework can outperform a state-of-the-art framework in terms of convergence speed.