ILT-Iterative LoRA Training through Focus-Feedback-Fix for Multilingual Speech Recognition
This addresses overfitting in fine-tuning large models for multilingual ASR, but appears incremental as it builds on existing LoRA and pseudo-labeling techniques.
The paper tackles overfitting in Low-Rank Adaptation (LoRA) for multilingual speech recognition by proposing Iterative LoRA Training (ILT) with an Iterative Pseudo Labeling strategy, achieving 4th and 1st place in Interspeech 2025 challenge tracks.
The deep integration of large language models and automatic speech recognition systems has become a promising research direction with high practical value. To address the overfitting issue commonly observed in Low-Rank Adaptation (LoRA) during the supervised fine-tuning (SFT) stage, this work proposes an innovative training paradigm Iterative LoRA Training (ILT) in combination with an Iterative Pseudo Labeling strategy, effectively enhancing the theoretical upper bound of model performance. Based on Whisper-large-v3 and Qwen2-Audio, we conduct systematic experiments using a three-stage training process: Focus Training, Feed Back Training, and Fix Training. Experimental results demonstrate the effectiveness of the proposed method. Furthermore, the MegaAIS research team applied this technique in the Interspeech 2025 Multilingual Conversational Speech Language Modeling Challenge (MLC-SLM), achieving 4th in Track 1 (Multilingual ASR Task) and 1st place in Track 2 (Speech Separation and Recognition Task), showcasing the practical feasibility and strong application potential of our approach.