Mind the Shift: Using Delta SSL Embeddings to Enhance Child ASR
This work addresses child ASR, a domain-specific problem with limited data, by proposing an incremental feature fusion method to improve performance.
The paper tackled the challenge of child automatic speech recognition (ASR) by using delta SSL embeddings, which are differences between finetuned and pretrained model embeddings, to complement finetuned features, resulting in up to a 10% relative WER reduction and a new state-of-the-art WER of 9.64 on the MyST corpus.
Self-supervised learning (SSL) models have achieved impressive results across many speech tasks, yet child automatic speech recognition (ASR) remains challenging due to limited data and pretraining domain mismatch. Fine-tuning SSL models on child speech induces shifts in the representation space. We hypothesize that delta SSL embeddings, defined as the differences between embeddings from a finetuned model and those from its pretrained counterpart, encode task-specific information that complements finetuned features from another SSL model. We evaluate multiple fusion strategies on the MyST childrens corpus using different models. Results show that delta embedding fusion with WavLM yields up to a 10 percent relative WER reduction for HuBERT and a 4.4 percent reduction for W2V2, compared to finetuned embedding fusion. Notably, fusing WavLM with delta W2V2 embeddings achieves a WER of 9.64, setting a new state of the art among SSL models on the MyST corpus. These findings demonstrate the effectiveness of delta embeddings and highlight feature fusion as a promising direction for advancing child ASR.