HiKE: Hierarchical Evaluation Framework for Korean-English Code-Switching Speech Recognition
It addresses a domain-specific problem for researchers in multilingual ASR by providing the first accessible evaluation framework for Korean-English code-switching, which is incremental as it builds on existing multilingual ASR methods.
The paper tackles the underexplored challenge of Korean-English code-switching in speech recognition by introducing HiKE, a benchmark framework that includes high-quality data and hierarchical labels, and shows that fine-tuning multilingual ASR models with synthetic data can enable this capability.
Despite advances in multilingual automatic speech recognition (ASR), code-switching (CS), the mixing of languages within an utterance common in daily speech, remains a severely underexplored challenge. In this paper, we introduce HiKE: the Hierarchical Korean-English code-switching benchmark, the first globally accessible evaluation framework for Korean-English CS, aiming to provide a means for the precise evaluation of multilingual ASR models and to foster research in the field. The proposed framework not only consists of high-quality, natural CS data across various topics, but also provides meticulous loanword labels and a hierarchical CS-level labeling scheme (word, phrase, and sentence) that together enable a systematic evaluation of a model's ability to handle each distinct level of code-switching. Through evaluations of diverse multilingual ASR models and fine-tuning experiments, this paper demonstrates that although most multilingual ASR models initially exhibit inadequate CS-ASR performance, this capability can be enabled through fine-tuning with synthetic CS data. HiKE is available at https://github.com/ThetaOne-AI/HiKE