QuPeL: Quantized Personalization with Applications to Federated Learning
This work addresses resource-efficient personalization for clients in federated learning, though it appears incremental as it builds on existing FL methods with quantization and personalization enhancements.
The paper tackles the challenges of data heterogeneity and resource diversity in federated learning by introducing QuPeL, a quantized and personalized algorithm that allows clients to learn compressed models with optimized quantization parameters, resulting in improved performance over FedAvg and local training in heterogeneous settings.
Traditionally, federated learning (FL) aims to train a single global model while collaboratively using multiple clients and a server. Two natural challenges that FL algorithms face are heterogeneity in data across clients and collaboration of clients with {\em diverse resources}. In this work, we introduce a \textit{quantized} and \textit{personalized} FL algorithm QuPeL that facilitates collective training with heterogeneous clients while respecting resource diversity. For personalization, we allow clients to learn \textit{compressed personalized models} with different quantization parameters depending on their resources. Towards this, first we propose an algorithm for learning quantized models through a relaxed optimization problem, where quantization values are also optimized over. When each client participating in the (federated) learning process has different requirements of the quantized model (both in value and precision), we formulate a quantized personalization framework by introducing a penalty term for local client objectives against a globally trained model to encourage collaboration. We develop an alternating proximal gradient update for solving this quantized personalization problem, and we analyze its convergence properties. Numerically, we show that optimizing over the quantization levels increases the performance and we validate that QuPeL outperforms both FedAvg and local training of clients in a heterogeneous setting.