Federated Acoustic Modeling For Automatic Speech Recognition
This work addresses data privacy for ASR service providers by enabling distributed model training without centralizing sensitive user data, offering an incremental improvement to federated learning for speech.
This paper explores federated acoustic modeling for automatic speech recognition (ASR) to address data privacy concerns. It proposes Client Adaptive Federated Training (CAFT) to mitigate non-IID data issues across clients, demonstrating its effectiveness in improving federated acoustic model performance on 1,150 hours of speech data.
Data privacy and protection is a crucial issue for any automatic speech recognition (ASR) service provider when dealing with clients. In this paper, we investigate federated acoustic modeling using data from multiple clients. A client's data is stored on a local data server and the clients communicate only model parameters with a central server, and not their data. The communication happens infrequently to reduce the communication cost. To mitigate the non-iid issue, client adaptive federated training (CAFT) is proposed to canonicalize data across clients. The experiments are carried out on 1,150 hours of speech data from multiple domains. Hybrid LSTM acoustic models are trained via federated learning and their performance is compared to traditional centralized acoustic model training. The experimental results demonstrate the effectiveness of the proposed federated acoustic modeling strategy. We also show that CAFT can further improve the performance of the federated acoustic model.