FiLM-Based Speaker Conditioning of a SpeechLLM for Pathological Speech Recognition
It addresses the challenge of adapting ASR to pathological speech for individuals with neurological conditions, offering a parameter-efficient alternative to fine-tuning.
The paper tackles pathological speech recognition by using FiLM-based speaker conditioning to adapt a frozen ASR encoder to individual speakers, achieving competitive performance with established adaptation methods while preserving the model's ability to answer speech-related questions.
Automatic speech recognition (ASR) has advanced remarkably for standard speech; however, pathological speech from neurological conditions remains a significant challenge. We investigate speaker conditioning via Feature-wise Linear Modulation (FiLM), injecting x-vector-derived information into each transformer layer of a frozen ASR encoder to adapt internal representations to individual pathological speakers without modifying base model weights. We benchmark this for the ASR task against standard and parameter-efficient fine-tuning baselines, complemented by post-processing, on Spanish and English pathological speech. Additionally, we evaluate if the adapted model preserves the ability to answer speech-related questions. Results show that speaker-conditioned ASR is competitive with established adaptation strategies while retaining performance on non-conditioned speech.