Diverse Semantic Image Synthesis via Probability Distribution Modeling
This addresses the problem of generating diverse, photo-realistic images from semantic layouts for applications in computer vision and graphics, representing a novel method for a known bottleneck.
The paper tackles the challenge of diverse semantic image synthesis by modeling semantic class distributions as continuous probability distributions, enabling multimodal generation at semantic or instance levels, and achieves superior diversity with comparable quality to state-of-the-art methods in experiments on multiple datasets.
Semantic image synthesis, translating semantic layouts to photo-realistic images, is a one-to-many mapping problem. Though impressive progress has been recently made, diverse semantic synthesis that can efficiently produce semantic-level multimodal results, still remains a challenge. In this paper, we propose a novel diverse semantic image synthesis framework from the perspective of semantic class distributions, which naturally supports diverse generation at semantic or even instance level. We achieve this by modeling class-level conditional modulation parameters as continuous probability distributions instead of discrete values, and sampling per-instance modulation parameters through instance-adaptive stochastic sampling that is consistent across the network. Moreover, we propose prior noise remapping, through linear perturbation parameters encoded from paired references, to facilitate supervised training and exemplar-based instance style control at test time. Extensive experiments on multiple datasets show that our method can achieve superior diversity and comparable quality compared to state-of-the-art methods. Code will be available at \url{https://github.com/tzt101/INADE.git}