Disentangling Timbre and Singing Style with Multi-singer Singing Synthesis System
This work addresses the challenge of creating more expressive and controllable singing synthesis for applications in music production and entertainment, representing an incremental improvement over existing methods.
The study tackled the problem of disentangling timbre and singing style in singing synthesis by proposing a multi-singer system that models them separately, and it experimentally verified through user listening tests that the framework generates high-quality, natural singing voices while enabling independent control of these aspects.
In this study, we define the identity of the singer with two independent concepts - timbre and singing style - and propose a multi-singer singing synthesis system that can model them separately. To this end, we extend our single-singer model into a multi-singer model in the following ways: first, we design a singer identity encoder that can adequately reflect the identity of a singer. Second, we use encoded singer identity to condition the two independent decoders that model timbre and singing style, respectively. Through a user study with the listening tests, we experimentally verify that the proposed framework is capable of generating a natural singing voice of high quality while independently controlling the timbre and singing style. Also, by using the method of changing singing styles while fixing the timbre, we suggest that our proposed network can produce a more expressive singing voice.