Spectral feature mapping with mimic loss for robust speech recognition
This addresses robust speech recognition in noisy environments, but it is incremental as it builds on existing methods by adding a global criterion.
The paper tackled the problem of speech enhancement being agnostic to phonetic structures for robust speech recognition by proposing a mimic loss to ensure de-noised speech is useful for downstream tasks, resulting in significant improvements in word error rate (WER) on the CHiME-2 corpus.
For the task of speech enhancement, local learning objectives are agnostic to phonetic structures helpful for speech recognition. We propose to add a global criterion to ensure de-noised speech is useful for downstream tasks like ASR. We first train a spectral classifier on clean speech to predict senone labels. Then, the spectral classifier is joined with our speech enhancer as a noisy speech recognizer. This model is taught to imitate the output of the spectral classifier alone on clean speech. This \textit{mimic loss} is combined with the traditional local criterion to train the speech enhancer to produce de-noised speech. Feeding the de-noised speech to an off-the-shelf Kaldi training recipe for the CHiME-2 corpus shows significant improvements in WER.