Enhancing Unsupervised Speech Recognition with Diffusion GANs
This work addresses unsupervised speech recognition for applications where labeled data is scarce, representing an incremental improvement over existing methods.
The paper tackles unsupervised automatic speech recognition by enhancing adversarial training with a diffusion-GAN approach, achieving word/phoneme error rates of 3.1% on Librispeech test-clean and 5.6% on test-other, outperforming wav2vec-U.
We enhance the vanilla adversarial training method for unsupervised Automatic Speech Recognition (ASR) by a diffusion-GAN. Our model (1) injects instance noises of various intensities to the generator's output and unlabeled reference text which are sampled from pretrained phoneme language models with a length constraint, (2) asks diffusion timestep-dependent discriminators to separate them, and (3) back-propagates the gradients to update the generator. Word/phoneme error rate comparisons with wav2vec-U under Librispeech (3.1% for test-clean and 5.6% for test-other), TIMIT and MLS datasets, show that our enhancement strategies work effectively.