Multimodal Adversarially Learned Inference with Factorized Discriminators
This work addresses the challenge of efficient multimodal representation learning for machine learning applications, though it appears incremental as it builds on existing GAN and contrastive learning frameworks.
The paper tackles the problem of learning coherent generative models from multimodal data by proposing a novel approach based on generative adversarial networks with factorized discriminators, and it reports outperforming state-of-the-art methods on benchmark datasets across various metrics.
Learning from multimodal data is an important research topic in machine learning, which has the potential to obtain better representations. In this work, we propose a novel approach to generative modeling of multimodal data based on generative adversarial networks. To learn a coherent multimodal generative model, we show that it is necessary to align different encoder distributions with the joint decoder distribution simultaneously. To this end, we construct a specific form of the discriminator to enable our model to utilize data efficiently, which can be trained constrastively. By taking advantage of contrastive learning through factorizing the discriminator, we train our model on unimodal data. We have conducted experiments on the benchmark datasets, whose promising results show that our proposed approach outperforms the-state-of-the-art methods on a variety of metrics. The source code will be made publicly available.