Modeling Singing F0 With Neural Network Driven Transition-Sustain Models
This addresses the problem of realistic singing synthesis for music production or voice synthesis applications, but it is incremental as it builds on prior neural network approaches for F0 modeling.
This study tackled the problem of generating fundamental frequency (F0) curves for singing voice from musical scores, where existing statistical models struggle with vibratos and note boundaries due to oversmoothing. The result was a neural network-based transition-sustain model that, in subjective listening tests on the NITech singing database, generated F0 trajectories close to the original performance.
This study focuses on generating fundamental frequency (F0) curves of singing voice from musical scores stored in a midi-like notation. Current statistical parametric approaches to singing F0 modeling meet difficulties in reproducing vibratos and the temporal details at note boundaries due to the oversmoothing tendency of statistical models. This paper presents a neural network based solution that models a pair of neighboring notes at a time (the transition model) and uses a separate network for generating vibratos (the sustain model). Predictions from the two models are combined by summation after proper enveloping to enforce continuity. In the training phase, mild misalignment between the scores and the target F0 is addressed by back-propagating the gradients to the networks' inputs. Subjective listening tests on the NITech singing database show that transition-sustain models are able to generate F0 trajectories close to the original performance.