Image Inspired Poetry Generation in XiaoIce
This addresses the challenge of creating artistic poetry from visual inspiration for users of AI systems like XiaoIce, representing an incremental advancement in AI-generated content.
The paper tackles the problem of generating poetry from images by extracting keywords for objects and sentiments, expanding them with associations from human poems, and using recurrent neural networks to create verses, resulting in poems rated as more artistic than baseline methods and over 12 million poems generated for users.
Vision is a common source of inspiration for poetry. The objects and the sentimental imprints that one perceives from an image may lead to various feelings depending on the reader. In this paper, we present a system of poetry generation from images to mimic the process. Given an image, we first extract a few keywords representing objects and sentiments perceived from the image. These keywords are then expanded to related ones based on their associations in human written poems. Finally, verses are generated gradually from the keywords using recurrent neural networks trained on existing poems. Our approach is evaluated by human assessors and compared to other generation baselines. The results show that our method can generate poems that are more artistic than the baseline methods. This is one of the few attempts to generate poetry from images. By deploying our proposed approach, XiaoIce has already generated more than 12 million poems for users since its release in July 2017. A book of its poems has been published by Cheers Publishing, which claimed that the book is the first-ever poetry collection written by an AI in human history.