EdiTTS: Score-based Editing for Controllable Text-to-Speech
This provides a practical tool for users needing granular speech editing without model modifications, though it is incremental as it builds on existing score-based models.
The paper tackles the problem of controllable speech editing in text-to-speech synthesis by introducing EdiTTS, a score-based method that enables targeted editing of audio content and pitch without retraining, and it outperforms existing baselines in listening tests and back transcription.
We present EdiTTS, an off-the-shelf speech editing methodology based on score-based generative modeling for text-to-speech synthesis. EdiTTS allows for targeted, granular editing of audio, both in terms of content and pitch, without the need for any additional training, task-specific optimization, or architectural modifications to the score-based model backbone. Specifically, we apply coarse yet deliberate perturbations in the Gaussian prior space to induce desired behavior from the diffusion model while applying masks and softening kernels to ensure that iterative edits are applied only to the target region. Through listening tests and speech-to-text back transcription, we show that EdiTTS outperforms existing baselines and produces robust samples that satisfy user-imposed requirements.