GILT: Generating Images from Long Text
This addresses a novel task for creative applications like book covers or posters, but it is incremental as it adapts an existing method to a new domain.
The paper tackled the problem of generating images from long, non-visual text, specifically creating 256x256 food images from recipes, using the recipe1M dataset and StackGAN-v2 architecture.
Creating an image reflecting the content of a long text is a complex process that requires a sense of creativity. For example, creating a book cover or a movie poster based on their summary or a food image based on its recipe. In this paper we present the new task of generating images from long text that does not describe the visual content of the image directly. For this, we build a system for generating high-resolution 256 $\times$ 256 images of food conditioned on their recipes. The relation between the recipe text (without its title) to the visual content of the image is vague, and the textual structure of recipes is complex, consisting of two sections (ingredients and instructions) both containing multiple sentences. We used the recipe1M dataset to train and evaluate our model that is based on a the StackGAN-v2 architecture.