Compositional GAN: Learning Image-Conditional Binary Composition
This addresses the challenge of explicit spatial composition in image generation for applications in computer vision and graphics, though it appears incremental as it builds on existing GAN frameworks.
The paper tackles the problem of generating realistic composite images from two distinct object distributions by capturing complex spatial interactions like scaling, layout, and occlusion, with results showing the model learns potential interactions and produces realistic scenes.
Generative Adversarial Networks (GANs) can produce images of remarkable complexity and realism but are generally structured to sample from a single latent source ignoring the explicit spatial interaction between multiple entities that could be present in a scene. Capturing such complex interactions between different objects in the world, including their relative scaling, spatial layout, occlusion, or viewpoint transformation is a challenging problem. In this work, we propose a novel self-consistent Composition-by-Decomposition (CoDe) network to compose a pair of objects. Given object images from two distinct distributions, our model can generate a realistic composite image from their joint distribution following the texture and shape of the input objects. We evaluate our approach through qualitative experiments and user evaluations. Our results indicate that the learned model captures potential interactions between the two object domains, and generates realistic composed scenes at test time.