Production federated keyword spotting via distillation, filtering, and joint federated-centralized training
This work addresses the problem of deploying efficient keyword spotting models on mobile devices for users, though it appears incremental by combining existing techniques like distillation and joint training.
The paper tackled the challenge of training a keyword spotting model using federated learning on real user devices, achieving significant improvements in offline quality metrics and user-experience metrics in live A/B experiments.
We trained a keyword spotting model using federated learning on real user devices and observed significant improvements when the model was deployed for inference on phones. To compensate for data domains that are missing from on-device training caches, we employed joint federated-centralized training. And to learn in the absence of curated labels on-device, we formulated a confidence filtering strategy based on user-feedback signals for federated distillation. These techniques created models that significantly improved quality metrics in offline evaluations and user-experience metrics in live A/B experiments.