Cross-domain Speech Recognition with Unsupervised Character-level Distribution Matching
This work addresses domain adaptation for speech recognition, which is an incremental improvement for handling mismatches between training and testing data.
The paper tackles the problem of domain mismatch in automatic speech recognition by proposing CMatch, an unsupervised domain adaptation method using character-level distribution matching, which achieved relative Word Error Rate reductions of 14.39% and 16.50% on cross-device and cross-environment tasks.
End-to-end automatic speech recognition (ASR) can achieve promising performance with large-scale training data. However, it is known that domain mismatch between training and testing data often leads to a degradation of recognition accuracy. In this work, we focus on the unsupervised domain adaptation for ASR and propose CMatch, a Character-level distribution matching method to perform fine-grained adaptation between each character in two domains. First, to obtain labels for the features belonging to each character, we achieve frame-level label assignment using the Connectionist Temporal Classification (CTC) pseudo labels. Then, we match the character-level distributions using Maximum Mean Discrepancy. We train our algorithm using the self-training technique. Experiments on the Libri-Adapt dataset show that our proposed approach achieves 14.39% and 16.50% relative Word Error Rate (WER) reduction on both cross-device and cross-environment ASR. We also comprehensively analyze the different strategies for frame-level label assignment and Transformer adaptations.