Three Approaches for Personalization with Applications to Federated Learning
This work addresses personalization in distributed learning settings, such as federated learning, but is incremental as it systematizes existing ideas with theoretical analysis.
The paper tackles the problem of learning personalized models per user in scenarios like federated learning, proposing three model-agnostic approaches (user clustering, data interpolation, and model interpolation) with learning-theoretic guarantees and efficient algorithms, and demonstrates their performance empirically.
The standard objective in machine learning is to train a single model for all users. However, in many learning scenarios, such as cloud computing and federated learning, it is possible to learn a personalized model per user. In this work, we present a systematic learning-theoretic study of personalization. We propose and analyze three approaches: user clustering, data interpolation, and model interpolation. For all three approaches, we provide learning-theoretic guarantees and efficient algorithms for which we also demonstrate the performance empirically. All of our algorithms are model-agnostic and work for any hypothesis class.