Wav2vec-C: A Self-supervised Model for Speech Representation Learning
This work addresses speech recognition by improving self-supervised learning with a novel hybrid method, showing incremental gains in a domain-specific context.
The paper tackles speech representation learning by introducing Wav2vec-C, a self-supervised model combining wav2vec 2.0 and VQ-VAE elements, which achieves an average twice the error reduction over baseline and higher codebook utilization compared to wav2vec 2.0.
Wav2vec-C introduces a novel representation learning technique combining elements from wav2vec 2.0 and VQ-VAE. Our model learns to reproduce quantized representations from partially masked speech encoding using a contrastive loss in a way similar to Wav2vec 2.0. However, the quantization process is regularized by an additional consistency network that learns to reconstruct the input features to the wav2vec 2.0 network from the quantized representations in a way similar to a VQ-VAE model. The proposed self-supervised model is trained on 10k hours of unlabeled data and subsequently used as the speech encoder in a RNN-T ASR model and fine-tuned with 1k hours of labeled data. This work is one of only a few studies of self-supervised learning on speech tasks with a large volume of real far-field labeled data. The Wav2vec-C encoded representations achieves, on average, twice the error reduction over baseline and a higher codebook utilization in comparison to wav2vec 2.0