Peking Opera Synthesis via Duration Informed Attention Network
This work addresses a domain-specific problem for preserving cultural heritage in Peking Opera synthesis, but it is incremental as it adapts existing methods to a new application.
The authors tackled the challenge of synthesizing expressive Peking Opera singing from music scores, where rhythm and pitch deviations from the score are common, by using a Duration Informed Attention Network with Lagrange multipliers and pseudo scores, resulting in high-quality timbre, pitch, and expressiveness.
Peking Opera has been the most dominant form of Chinese performing art since around 200 years ago. A Peking Opera singer usually exhibits a very strong personal style via introducing improvisation and expressiveness on stage which leads the actual rhythm and pitch contour to deviate significantly from the original music score. This inconsistency poses a great challenge in Peking Opera singing voice synthesis from a music score. In this work, we propose to deal with this issue and synthesize expressive Peking Opera singing from the music score based on the Duration Informed Attention Network (DurIAN) framework. To tackle the rhythm mismatch, Lagrange multiplier is used to find the optimal output phoneme duration sequence with the constraint of the given note duration from music score. As for the pitch contour mismatch, instead of directly inferring from music score, we adopt a pseudo music score generated from the real singing and feed it as input during training. The experiments demonstrate that with the proposed system we can synthesize Peking Opera singing voice with high-quality timbre, pitch and expressiveness.