Federated Evaluation of On-device Personalization
This work addresses the challenge of assessing personalization effectiveness in federated settings for users of on-device applications, but it is incremental as it extends existing federation frameworks.
The paper tackled the problem of evaluating personalization strategies for global models within federated learning, reporting that a significant fraction of users benefit from personalization in experiments with a language model for a smartphone keyboard across tens of millions of users.
Federated learning is a distributed, on-device computation framework that enables training global models without exporting sensitive user data to servers. In this work, we describe methods to extend the federation framework to evaluate strategies for personalization of global models. We present tools to analyze the effects of personalization and evaluate conditions under which personalization yields desirable models. We report on our experiments personalizing a language model for a virtual keyboard for smartphones with a population of tens of millions of users. We show that a significant fraction of users benefit from personalization.