Voice Conversion Based Speaker Normalization for Acoustic Unit Discovery
This addresses the challenge of speaker-independent acoustic unit discovery for speech processing, particularly in low-resource or multilingual settings, though it is incremental as it builds on existing unit discovery systems.
The paper tackles the problem of discovering speaker-independent acoustic units from spoken input by proposing an unsupervised speaker normalization technique that separates speaker and content variations using adversarial contrastive predictive coding. Experiments on English, Yoruba, and Mboshi show improvements in acoustic unit discovery compared to non-normalized input.
Discovering speaker independent acoustic units purely from spoken input is known to be a hard problem. In this work we propose an unsupervised speaker normalization technique prior to unit discovery. It is based on separating speaker related from content induced variations in a speech signal with an adversarial contrastive predictive coding approach. This technique does neither require transcribed speech nor speaker labels, and, furthermore, can be trained in a multilingual fashion, thus achieving speaker normalization even if only few unlabeled data is available from the target language. The speaker normalization is done by mapping all utterances to a medoid style which is representative for the whole database. We demonstrate the effectiveness of the approach by conducting acoustic unit discovery with a hidden Markov model variational autoencoder noting, however, that the proposed speaker normalization can serve as a front end to any unit discovery system. Experiments on English, Yoruba and Mboshi show improvements compared to using non-normalized input.