Unsupervised Adaptation with Interpretable Disentangled Representations for Distant Conversational Speech Recognition
This addresses the costly need for labeled data in speech recognition for domain adaptation, particularly in channel mismatch scenarios like distant conversational speech, though it is incremental as it builds on existing unsupervised and disentanglement techniques.
The paper tackles the problem of adapting speech recognition models to new domains without labeled data by learning interpretable disentangled representations to synthesize labeled target-domain data from unlabeled in-domain and labeled out-of-domain data, achieving a 77% reduction in the performance gap between unadapted and in-domain models on the AMI dataset.
The current trend in automatic speech recognition is to leverage large amounts of labeled data to train supervised neural network models. Unfortunately, obtaining data for a wide range of domains to train robust models can be costly. However, it is relatively inexpensive to collect large amounts of unlabeled data from domains that we want the models to generalize to. In this paper, we propose a novel unsupervised adaptation method that learns to synthesize labeled data for the target domain from unlabeled in-domain data and labeled out-of-domain data. We first learn without supervision an interpretable latent representation of speech that encodes linguistic and nuisance factors (e.g., speaker and channel) using different latent variables. To transform a labeled out-of-domain utterance without altering its transcript, we transform the latent nuisance variables while maintaining the linguistic variables. To demonstrate our approach, we focus on a channel mismatch setting, where the domain of interest is distant conversational speech, and labels are only available for close-talking speech. Our proposed method is evaluated on the AMI dataset, outperforming all baselines and bridging the gap between unadapted and in-domain models by over 77% without using any parallel data.