Multilingual Adaptation of RNN Based ASR Systems
This work addresses the problem of multilingual ASR adaptation for researchers and practitioners, offering an incremental improvement over previous methods.
The paper tackled the challenge of training multilingual automatic speech recognition (ASR) systems with recurrent neural networks (RNNs) by proposing a modulation technique using Language Feature Vectors (LFVs) to adapt deeper network layers, resulting in lower error rates across full and low resource conditions for grapheme and phone-based systems.
In this work, we focus on multilingual systems based on recurrent neural networks (RNNs), trained using the Connectionist Temporal Classification (CTC) loss function. Using a multilingual set of acoustic units poses difficulties. To address this issue, we proposed Language Feature Vectors (LFVs) to train language adaptive multilingual systems. Language adaptation, in contrast to speaker adaptation, needs to be applied not only on the feature level, but also to deeper layers of the network. In this work, we therefore extended our previous approach by introducing a novel technique which we call "modulation". Based on this method, we modulated the hidden layers of RNNs using LFVs. We evaluated this approach in both full and low resource conditions, as well as for grapheme and phone based systems. Lower error rates throughout the different conditions could be achieved by the use of the modulation.