Analysis of Multilingual Sequence-to-Sequence speech recognition systems
This work addresses multilingual ASR for low-resource languages, but it is incremental as it adapts existing HMM methods to seq2seq systems.
This paper tackled the problem of applying multilingual techniques from HMM systems to seq2seq ASR, finding that multilingual features outperform multilingual models and enabling efficient combination of HMM and seq2seq benefits.
This paper investigates the applications of various multilingual approaches developed in conventional hidden Markov model (HMM) systems to sequence-to-sequence (seq2seq) automatic speech recognition (ASR). On a set composed of Babel data, we first show the effectiveness of multi-lingual training with stacked bottle-neck (SBN) features. Then we explore various architectures and training strategies of multi-lingual seq2seq models based on CTC-attention networks including combinations of output layer, CTC and/or attention component re-training. We also investigate the effectiveness of language-transfer learning in a very low resource scenario when the target language is not included in the original multi-lingual training data. Interestingly, we found multilingual features superior to multilingual models, and this finding suggests that we can efficiently combine the benefits of the HMM system with the seq2seq system through these multilingual feature techniques.