Federated Representation Learning for Automatic Speech Recognition
This work addresses privacy concerns in speech recognition for edge device users by enabling collaborative learning without data sharing, though it is incremental as it applies existing methods to a new context.
The paper tackles the problem of learning robust audio representations for automatic speech recognition while preserving data privacy by combining federated learning with self-supervised learning. The result shows that the federated pre-trained model performs as well as a centrally trained model, achieving a 12-15% word error rate improvement over no pre-training and a 20% improvement when adapted to a new language.
Federated Learning (FL) is a privacy-preserving paradigm, allowing edge devices to learn collaboratively without sharing data. Edge devices like Alexa and Siri are prospective sources of unlabeled audio data that can be tapped to learn robust audio representations. In this work, we bring Self-supervised Learning (SSL) and FL together to learn representations for Automatic Speech Recognition respecting data privacy constraints. We use the speaker and chapter information in the unlabeled speech dataset, Libri-Light, to simulate non-IID speaker-siloed data distributions and pre-train an LSTM encoder with the Contrastive Predictive Coding framework with FedSGD. We show that the pre-trained ASR encoder in FL performs as well as a centrally pre-trained model and produces an improvement of 12-15% (WER) compared to no pre-training. We further adapt the federated pre-trained models to a new language, French, and show a 20% (WER) improvement over no pre-training.