Adversarially Trained Multi-Singer Sequence-To-Sequence Singing Synthesizer
This work addresses the challenge of generating realistic singing voices for applications like music production and entertainment, though it is incremental as it builds on existing sequence-to-sequence and GAN methods.
The paper tackles the problem of synthesizing high-quality singing voices with limited recordings by introducing a multi-singer sequence-to-sequence framework with adversarial training and multiple random window discriminators, achieving a mean opinion score (MOS) of 4.12 compared to a baseline of 3.53.
This paper presents a high quality singing synthesizer that is able to model a voice with limited available recordings. Based on the sequence-to-sequence singing model, we design a multi-singer framework to leverage all the existing singing data of different singers. To attenuate the issue of musical score unbalance among singers, we incorporate an adversarial task of singer classification to make encoder output less singer dependent. Furthermore, we apply multiple random window discriminators (MRWDs) on the generated acoustic features to make the network be a GAN. Both objective and subjective evaluations indicate that the proposed synthesizer can generate higher quality singing voice than baseline (4.12 vs 3.53 in MOS). Especially, the articulation of high-pitched vowels is significantly enhanced.