CLAR: CIF-Localized Alignment for Retrieval-Augmented Speech LLM-Based Contextual ASR
This addresses accuracy issues in contextual ASR for speech recognition systems, particularly for named entities, but is incremental as it builds on existing retrieval-augmented biasing methods.
The paper tackles the problem of named entity and long-tail word recognition in speech LLM-based ASR by proposing CLAR, a retrieval-augmented method that uses CIF for token-level alignments without timestamps, resulting in significant improvements in hotword retrieval and reductions in CER and B-WER against baselines.
Speech LLM-based ASR often struggles with named entities and long-tail words due to strong internal language-model priors. Retrieval-augmented biasing can help, but its effectiveness depends on accurate hotword localization in full-utterance speech under weak supervision. We propose CLAR, a dual-encoder speech-text retriever that uses Continuous Integrate-and-Fire (CIF) to learn monotonic token-level alignments without timestamps. With length-aware localized matching, CLAR anchors short-entity acoustic cues and reduces representation dilution and attention drift. The retriever is trained with a multi-granularity objective combining global and local segment-level contrastive losses and a CIF quantity constraint. At inference, top-ranked hotwords are injected as contextual prompts for the Speech LLM, improving recognition without shallow fusion. Experiments show that CLAR significantly improves hotword retrieval and reduces both CER and B-WER against strong contextual ASR baselines.