Transformer VQ-VAE for Unsupervised Unit Discovery and Speech Synthesis: ZeroSpeech 2020 Challenge
This addresses the problem of building speech synthesizers without phonetic labels for applications in low-resource or unsupervised speech processing, though it is incremental as it adapts existing methods to a specific challenge.
The paper tackled unsupervised speech synthesis without textual labels by proposing a Transformer-based VQ-VAE for unit discovery and a Transformer-based inverter for synthesis, achieving results for the ZeroSpeech 2020 challenge that balanced ABX error rate and bitrate compression.
In this paper, we report our submitted system for the ZeroSpeech 2020 challenge on Track 2019. The main theme in this challenge is to build a speech synthesizer without any textual information or phonetic labels. In order to tackle those challenges, we build a system that must address two major components such as 1) given speech audio, extract subword units in an unsupervised way and 2) re-synthesize the audio from novel speakers. The system also needs to balance the codebook performance between the ABX error rate and the bitrate compression rate. Our main contribution here is we proposed Transformer-based VQ-VAE for unsupervised unit discovery and Transformer-based inverter for the speech synthesis given the extracted codebook. Additionally, we also explored several regularization methods to improve performance even further.