Multimodal Image Synthesis with Conditional Implicit Maximum Likelihood Estimation
This addresses the mode collapse problem in generative adversarial nets for computer vision and graphics applications, offering a generic solution for multimodal image synthesis.
The paper tackles the challenge of generating diverse and plausible images in conditional image synthesis by developing a new multimodal method based on Implicit Maximum Likelihood Estimation, demonstrating improved performance on tasks like single image super-resolution and image synthesis from scene layouts.
Many tasks in computer vision and graphics fall within the framework of conditional image synthesis. In recent years, generative adversarial nets (GANs) have delivered impressive advances in quality of synthesized images. However, it remains a challenge to generate both diverse and plausible images for the same input, due to the problem of mode collapse. In this paper, we develop a new generic multimodal conditional image synthesis method based on Implicit Maximum Likelihood Estimation (IMLE) and demonstrate improved multimodal image synthesis performance on two tasks, single image super-resolution and image synthesis from scene layouts. We make our implementation publicly available.