Text2Scene: Generating Compositional Scenes from Textual Descriptions
This addresses the problem of controllable and interpretable scene generation from text for applications in computer vision and graphics, though it is incremental as it builds on existing text-to-scene methods with a novel sequential approach.
The paper tackles the problem of generating compositional scene representations from text descriptions by proposing Text2Scene, a model that sequentially generates objects and their attributes without using GANs. The result shows it is competitive with GAN-based methods on automatic metrics, superior in human judgments, and produces interpretable outputs across cartoon scenes, object layouts, and synthetic images.
In this paper, we propose Text2Scene, a model that generates various forms of compositional scene representations from natural language descriptions. Unlike recent works, our method does NOT use Generative Adversarial Networks (GANs). Text2Scene instead learns to sequentially generate objects and their attributes (location, size, appearance, etc) at every time step by attending to different parts of the input text and the current status of the generated scene. We show that under minor modifications, the proposed framework can handle the generation of different forms of scene representations, including cartoon-like scenes, object layouts corresponding to real images, and synthetic images. Our method is not only competitive when compared with state-of-the-art GAN-based methods using automatic metrics and superior based on human judgments but also has the advantage of producing interpretable results.