MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets
This addresses the computational demands of GANs for distributed data scenarios, offering a practical improvement over existing methods.
The paper tackles the problem of training Generative Adversarial Networks (GANs) on distributed datasets across multiple workers, proposing MD-GAN as a novel solution that reduces learning complexity by a factor of two on each worker node and outperforms federated learning on MNIST and CIFAR10 datasets.
A recent technical breakthrough in the domain of machine learning is the discovery and the multiple applications of Generative Adversarial Networks (GANs). Those generative models are computationally demanding, as a GAN is composed of two deep neural networks, and because it trains on large datasets. A GAN is generally trained on a single server. In this paper, we address the problem of distributing GANs so that they are able to train over datasets that are spread on multiple workers. MD-GAN is exposed as the first solution for this problem: we propose a novel learning procedure for GANs so that they fit this distributed setup. We then compare the performance of MD-GAN to an adapted version of Federated Learning to GANs, using the MNIST and CIFAR10 datasets. MD-GAN exhibits a reduction by a factor of two of the learning complexity on each worker node, while providing better performances than federated learning on both datasets. We finally discuss the practical implications of distributing GANs.