A Correspondence Variational Autoencoder for Unsupervised Acoustic Word Embeddings
This work addresses the problem of creating acoustic word embeddings for search, discovery, and indexing systems, particularly benefiting low- and zero-resource languages where labeled data is scarce.
This paper introduces a new unsupervised model, the maximal sampling correspondence variational autoencoder (MCVAE), for generating fixed-dimensional acoustic word embeddings from variable-duration speech segments. The MCVAE outperforms previous state-of-the-art models in both zero-resource and semi-supervised low-resource settings.
We propose a new unsupervised model for mapping a variable-duration speech segment to a fixed-dimensional representation. The resulting acoustic word embeddings can form the basis of search, discovery, and indexing systems for low- and zero-resource languages. Our model, which we refer to as a maximal sampling correspondence variational autoencoder (MCVAE), is a recurrent neural network (RNN) trained with a novel self-supervised correspondence loss that encourages consistency between embeddings of different instances of the same word. Our training scheme improves on previous correspondence training approaches through the use and comparison of multiple samples from the approximate posterior distribution. In the zero-resource setting, the MCVAE can be trained in an unsupervised way, without any ground-truth word pairs, by using the word-like segments discovered via an unsupervised term discovery system. In both this setting and a semi-supervised low-resource setting (with a limited set of ground-truth word pairs), the MCVAE outperforms previous state-of-the-art models, such as Siamese-, CAE- and VAE-based RNNs.