Referenceless User Controllable Semantic Image Synthesis
This work addresses the challenge of user controllability in image generation for applications like design and editing, though it is incremental in improving style flexibility.
The paper tackles the problem of limited style control in semantic image synthesis by proposing RUCGAN, a model that uses a single color per semantic region to achieve reference-free synthesis, enabling unusual color styles and outperforming existing methods in experiments.
Despite recent progress in semantic image synthesis, complete control over image style remains a challenging problem. Existing methods require reference images to feed style information into semantic layouts, which indicates that the style is constrained by the given image. In this paper, we propose a model named RUCGAN for user controllable semantic image synthesis, which utilizes a singular color to represent the style of a specific semantic region. The proposed network achieves reference-free semantic image synthesis by injecting color as user-desired styles into each semantic layout, and is able to synthesize semantic images with unusual colors. Extensive experimental results on various challenging datasets show that the proposed method outperforms existing methods, and we further provide an interactive UI to demonstrate the advantage of our approach for style controllability.