Automated Evaluation of Meter and Rhyme in Russian Generative and Human-Authored Poetry
This work addresses the need for automated evaluation tools in creative generative AI, specifically for Russian poetry, but it is incremental as it builds on existing versification analysis methods.
The authors tackled the problem of evaluating meter and rhyme in Russian poetry by introducing a library for stress mark placement and rhyme detection, along with releasing an annotated dataset of poem fragments. The result is a set of tools and data that can be used to assess generative poetry systems and large language models.
Generative poetry systems require effective tools for data engineering and automatic evaluation, particularly to assess how well a poem adheres to versification rules, such as the correct alternation of stressed and unstressed syllables and the presence of rhymes. In this work, we introduce the Russian Poetry Scansion Tool library designed for stress mark placement in Russian-language syllabo-tonic poetry, rhyme detection, and identification of defects of poeticness. Additionally, we release RIFMA -- a dataset of poem fragments spanning various genres and forms, annotated with stress marks. This dataset can be used to evaluate the capability of modern large language models to accurately place stress marks in poetic texts. The published resources provide valuable tools for researchers and practitioners in the field of creative generative AI, facilitating advancements in the development and evaluation of generative poetry systems.