Semi-supervised Chinese Poem-to-Painting Generation via Cycle-consistent Adversarial Networks
This addresses the problem of artistic translation between poetry and painting for cultural and computational domains, but it is incremental as it builds on existing adversarial network techniques.
The paper tackled the challenge of generating paintings from Chinese poems with limited paired data by proposing a semi-supervised approach using cycle-consistent adversarial networks, which outperformed previous methods on a new dataset.
Classical Chinese poetry and painting represent the epitome of artistic expression, but the abstract and symbolic nature of their relationship poses a significant challenge for computational translation. Most existing methods rely on large-scale paired datasets, which are scarce in this domain. In this work, we propose a semi-supervised approach using cycle-consistent adversarial networks to leverage the limited paired data and large unpaired corpus of poems and paintings. The key insight is to learn bidirectional mappings that enforce semantic alignment between the visual and textual modalities. We introduce novel evaluation metrics to assess the quality, diversity, and consistency of the generated poems and paintings. Extensive experiments are conducted on a new Chinese Painting Description Dataset (CPDD). The proposed model outperforms previous methods, showing promise in capturing the symbolic essence of artistic expression. Codes are available online \url{https://github.com/Mnster00/poemtopainting}.