Language Family Matters: Evaluating LLM-Based ASR Across Linguistic Boundaries
This provides a practical and scalable strategy for multilingual ASR deployment, though it is incremental.
The paper tackled the problem of training separate connectors per language for LLM-based ASR systems, which overlooks linguistic relatedness, and found that using family-based connectors reduces parameters while improving generalization across domains.
Large Language Model (LLM)-powered Automatic Speech Recognition (ASR) systems achieve strong performance with limited resources by linking a frozen speech encoder to a pretrained LLM via a lightweight connector. Prior work trains a separate connector per language, overlooking linguistic relatedness. We propose an efficient and novel connector-sharing strategy based on linguistic family membership, enabling one connector per family, and empirically validate its effectiveness across two multilingual LLMs and two real-world corpora spanning curated and crowd-sourced speech. Our results show that family-based connectors reduce parameter count while improving generalization across domains, offering a practical and scalable strategy for multilingual ASR deployment.