Improving Semi-supervised End-to-end Automatic Speech Recognition using CycleGAN and Inter-domain Losses
This work addresses speech recognition for domains with limited paired data, but it is incremental as it builds on existing CycleGAN and inter-domain loss techniques.
The paper tackled the problem of semi-supervised end-to-end automatic speech recognition by combining CycleGAN and inter-domain losses to improve shared representation from unpaired speech-text inputs, resulting in an 8-8.5% character error rate reduction on WSJ eval92 and Voxforge datasets and noticeable improvements on LibriSpeech.
We propose a novel method that combines CycleGAN and inter-domain losses for semi-supervised end-to-end automatic speech recognition. Inter-domain loss targets the extraction of an intermediate shared representation of speech and text inputs using a shared network. CycleGAN uses cycle-consistent loss and the identity mapping loss to preserve relevant characteristics of the input feature after converting from one domain to another. As such, both approaches are suitable to train end-to-end models on unpaired speech-text inputs. In this paper, we exploit the advantages from both inter-domain loss and CycleGAN to achieve better shared representation of unpaired speech and text inputs and thus improve the speech-to-text mapping. Our experimental results on the WSJ eval92 and Voxforge (non English) show 8~8.5% character error rate reduction over the baseline, and the results on LibriSpeech test_clean also show noticeable improvement.