Towards Personalized Federated Learning for Dysarthric Speech Recognition
For researchers working on federated learning for dysarthric speech recognition, this work provides personalized aggregation strategies that improve performance over a standard baseline.
The paper addresses the challenge of speech recognition for dysarthric speakers using federated learning with personalization. The proposed aggregation strategies achieve statistically significant word error rate reductions of up to 0.99% absolute (3.15% relative) on UASpeech and 0.56% absolute (4.73% relative) on TORGO compared to baseline regularized FedAvg.
Speech recognition is challenging for dysarthric speakers. While federated learning (FL)-based ASR can be an effective tool for protecting privacy, it suffers from heterogeneity issues caused by speaker variability. Forcing all speakers to share the same model components can be suboptimal under such heterogeneity, making personalization a promising direction; however, related research on dysarthric speech remains limited. To this end, this paper explores two aggregation strategies to achieve personalization, including the parameter-based averaging strategy and the embedding-based averaging strategy. Experiments on UASpeech and TORGO show that the proposed methods outperform the baseline regularized FedAvg by statistically significant WER reductions of up to 0.99% absolute (3.15% relative) on UASpeech and 0.56% absolute (4.73% relative) on TORGO, respectively.