Lightweight and perceptually-guided voice conversion for electro-laryngeal speech
This work addresses voice rehabilitation for electro-laryngeal speech users, but it is incremental as it adapts an existing framework to a specific domain.
The paper tackled the problem of improving naturalness and intelligibility in electro-laryngeal speech by adapting a lightweight voice conversion framework, resulting in a character error rate reduction, naturalness score increase from 1.1 to 3.3, and narrowed performance gaps to healthy speech.
Electro-laryngeal (EL) speech is characterized by constant pitch, limited prosody, and mechanical noise, reducing naturalness and intelligibility. We propose a lightweight adaptation of the state-of-the-art StreamVC framework to this setting by removing pitch and energy modules and combining self-supervised pretraining with supervised fine-tuning on parallel EL and healthy (HE) speech data, guided by perceptual and intelligibility losses. Objective and subjective evaluations across different loss configurations confirm their influence: the best model variant, based on WavLM features and human-feedback predictions (+WavLM+HF), drastically reduces character error rate (CER) of EL inputs, raises naturalness mean opinion score (nMOS) from 1.1 to 3.3, and consistently narrows the gap to HE ground-truth speech in all evaluated metrics. These findings demonstrate the feasibility of adapting lightweight voice conversion architectures to EL voice rehabilitation while also identifying prosody generation and intelligibility improvements as the main remaining bottlenecks.