Modern French Poetry Generation with RoBERTa and GPT-2
This work addresses poetry generation for French language applications, but it is incremental as it adapts existing models to a new domain.
The authors tackled modern French poetry generation by fine-tuning a hybrid model combining RoBERTa for understanding and GPT-2 for generation, achieving human evaluation scores ranging from 3.57 to 3.79 on a 5-point scale for criteria like typicality and understandability.
We present a novel neural model for modern poetry generation in French. The model consists of two pretrained neural models that are fine-tuned for the poem generation task. The encoder of the model is a RoBERTa based one while the decoder is based on GPT-2. This way the model can benefit from the superior natural language understanding performance of RoBERTa and the good natural language generation performance of GPT-2. Our evaluation shows that the model can create French poetry successfully. On a 5 point scale, the lowest score of 3.57 was given by human judges to typicality and emotionality of the output poetry while the best score of 3.79 was given to understandability.