PoeticTTS -- Controllable Poetry Reading for Literary Studies
This work addresses the problem of enabling literary scholars to systematically examine hypotheses about poetry recitation, though it is incremental as it builds on existing TTS methods with specific adaptations.
The authors tackled the challenge of synthesizing poetry with natural intonation patterns to aid literary studies, achieving results that captured poetic intonation to a large extent and were verified through objective and human evaluations.
Speech synthesis for poetry is challenging due to specific intonation patterns inherent to poetic speech. In this work, we propose an approach to synthesise poems with almost human like naturalness in order to enable literary scholars to systematically examine hypotheses on the interplay between text, spoken realisation, and the listener's perception of poems. To meet these special requirements for literary studies, we resynthesise poems by cloning prosodic values from a human reference recitation, and afterwards make use of fine-grained prosody control to manipulate the synthetic speech in a human-in-the-loop setting to alter the recitation w.r.t. specific phenomena. We find that finetuning our TTS model on poetry captures poetic intonation patterns to a large extent which is beneficial for prosody cloning and manipulation and verify the success of our approach both in an objective evaluation as well as in human studies.