Paint it Black: Generating paintings from text descriptions
This work addresses a niche problem in AI art generation for applications in creative industries, but it is incremental as it combines existing methods for text-to-image and style transfer.
The paper tackles the problem of generating paintings from text descriptions, an underexplored area with limited data, by exploring and integrating two strategies: generating photorealistic images followed by style transfer, and training an image generation model on real images with captions then fine-tuning on captioned paintings. The results are evaluated using metrics and a user study for human feedback.
Two distinct tasks - generating photorealistic pictures from given text prompts and transferring the style of a painting to a real image to make it appear as though it were done by an artist, have been addressed many times, and several approaches have been proposed to accomplish them. However, the intersection of these two, i.e., generating paintings from a given caption, is a relatively unexplored area with little data available. In this paper, we have explored two distinct strategies and have integrated them together. First strategy is to generate photorealistic images and then apply style transfer and the second strategy is to train an image generation model on real images with captions and then fine-tune it on captioned paintings later. These two models are evaluated using different metrics as well as a user study is conducted to get human feedback on the produced results.