MEGAN: Mixture of Experts of Generative Adversarial Networks for Multimodal Image Generation
This addresses the issue of poor image quality in multimodal generation for researchers and practitioners in computer vision, though it is incremental as it builds on existing GAN and mixture-of-experts frameworks.
The paper tackles the problem of GANs struggling to learn complex multimodal data by proposing MEGAN, a mixture of experts approach that uses multiple specialized generators, achieving an MS-SSIM score of 0.2470 on CelebA and an unsupervised inception score of 8.33 on CIFAR-10.
Recently, generative adversarial networks (GANs) have shown promising performance in generating realistic images. However, they often struggle in learning complex underlying modalities in a given dataset, resulting in poor-quality generated images. To mitigate this problem, we present a novel approach called mixture of experts GAN (MEGAN), an ensemble approach of multiple generator networks. Each generator network in MEGAN specializes in generating images with a particular subset of modalities, e.g., an image class. Instead of incorporating a separate step of handcrafted clustering of multiple modalities, our proposed model is trained through an end-to-end learning of multiple generators via gating networks, which is responsible for choosing the appropriate generator network for a given condition. We adopt the categorical reparameterization trick for a categorical decision to be made in selecting a generator while maintaining the flow of the gradients. We demonstrate that individual generators learn different and salient subparts of the data and achieve a multiscale structural similarity (MS-SSIM) score of 0.2470 for CelebA and a competitive unsupervised inception score of 8.33 in CIFAR-10.