Linear networks based speaker adaptation for speech synthesis
This work addresses the problem of creating high-quality synthetic voices for target speakers with minimal data, which is incremental in improving adaptation efficiency in speech synthesis.
The paper tackles speaker adaptation for speech synthesis with limited data by inserting linear networks into neural network layers and using low-rank plus diagonal decomposition for stability. Their best model with only 200 adaptation utterances achieves comparable quality to a speaker-dependent model trained with 1000 utterances in naturalness and similarity.
Speaker adaptation methods aim to create fair quality synthesis speech voice font for target speakers while only limited resources available. Recently, as deep neural networks based statistical parametric speech synthesis (SPSS) methods become dominant in SPSS TTS back-end modeling, speaker adaptation under the neural network based SPSS framework has also became an important task. In this paper, linear networks (LN) is inserted in multiple neural network layers and fine-tuned together with output layer for best speaker adaptation performance. When adaptation data is extremely small, the low-rank plus diagonal(LRPD) decomposition for LN is employed to make the adapted voice more stable. Speaker adaptation experiments are conducted under a range of adaptation utterances numbers. Moreover, speaker adaptation from 1) female to female, 2) male to female and 3) female to male are investigated. Objective measurement and subjective tests show that LN with LRPD decomposition performs most stable when adaptation data is extremely limited, and our best speaker adaptation (SA) model with only 200 adaptation utterances achieves comparable quality with speaker dependent (SD) model trained with 1000 utterances, in both naturalness and similarity to target speaker.