Sequence-to-sequence Singing Voice Synthesis with Perceptual Entropy Loss
This work addresses data scarcity and over-fitting issues in singing voice synthesis, which is a domain-specific problem for audio synthesis researchers and developers, and it is incremental as it applies a novel regularization technique to existing models.
The authors tackled the problem of over-fitting in neural network-based singing voice synthesis (SVS) systems due to data scarcity by proposing a Perceptual Entropy (PE) loss derived from a psycho-acoustic hearing model. Their experiments on a one-hour open-source database showed that the PE loss mitigates over-fitting and significantly improves synthesized singing quality in objective and subjective evaluations.
The neural network (NN) based singing voice synthesis (SVS) systems require sufficient data to train well and are prone to over-fitting due to data scarcity. However, we often encounter data limitation problem in building SVS systems because of high data acquisition and annotation costs. In this work, we propose a Perceptual Entropy (PE) loss derived from a psycho-acoustic hearing model to regularize the network. With a one-hour open-source singing voice database, we explore the impact of the PE loss on various mainstream sequence-to-sequence models, including the RNN-based, transformer-based, and conformer-based models. Our experiments show that the PE loss can mitigate the over-fitting problem and significantly improve the synthesized singing quality reflected in objective and subjective evaluations.