Continual Learning in Machine Speech Chain Using Gradient Episodic Memory
This addresses the challenge of maintaining ASR performance on old tasks while learning new ones, though it appears incremental as it builds on existing continual learning techniques.
The paper tackled the problem of catastrophic forgetting in continual learning for automatic speech recognition by integrating a text-to-speech component into the machine speech chain to support gradient episodic memory, resulting in a substantial error rate reduction compared to traditional methods on the LJ Speech dataset.
Continual learning for automatic speech recognition (ASR) systems poses a challenge, especially with the need to avoid catastrophic forgetting while maintaining performance on previously learned tasks. This paper introduces a novel approach leveraging the machine speech chain framework to enable continual learning in ASR using gradient episodic memory (GEM). By incorporating a text-to-speech (TTS) component within the machine speech chain, we support the replay mechanism essential for GEM, allowing the ASR model to learn new tasks sequentially without significant performance degradation on earlier tasks. Our experiments, conducted on the LJ Speech dataset, demonstrate that our method outperforms traditional fine-tuning and multitask learning approaches, achieving a substantial error rate reduction while maintaining high performance across varying noise conditions. We showed the potential of our semi-supervised machine speech chain approach for effective and efficient continual learning in speech recognition.